x86 Fast Reboot will be available in snv_100

Table of Contents

============================================================================
1. Usage Model
    1.1. Dry run
    1.2. Fast reboot without arguments
    1.3. Fast reboot with unix arguments
    1.4. Fast reboot with root disk or pool arguments
    1.5. Fast reboot with -e option
    1.6. Fast reboot with alternate mounted root file system

2. Known-to-work Drivers
    2.1 NIC Drivers
    2.2 HBA Drivers
    2.3 USB Drivers

3. Known-to-work Platforms
    3.1 Sun Fire Servers
    3.2 Sun Blade Servers
    3.3 Desktops
    3.4 Other

4. How To Test Your quiesce(9E)

5. Known Issues
    5.1 ON bugs you might run into
    5.2 Possible conditions where fast reboot would fail
        5.2.1 Not all drivers have implemented quiesce(9E)
        5.2.2 Not enough memory
        5.2.3 Not supported

6. Features (Not Bugs)
    6.1 Transient menu entry
    6.2 Limited functionality for "uadmin 2 8"

7. A little secret

============================================================================


1. Usage Model

    1.1. Dry run: this is an undocumented internal command to check
         whether the fast reboot attempt will likely to succeed.

        # reboot -f dryrun

        On success:

            # reboot -f dryrun
            reboot: all drivers have implemented quiesce(9E)

        On failure:

            # reboot -f dryrun
            (Drivers without quiesce() implementation will be listed)
             genunix: WARNING: nvidia has no quiesce()
             reboot: not all drivers have implemented quiesce(9E)

            If you don't have console access, you can get the list of
            drivers without quiesce(9E) support like this:

            # grep "no quiesce" /var/adm/messages

    1.2. Fast reboot without arguments

        # reboot -f
        # uadmin 2 8

        NOTE: Whenever "uadmin" is used, you must make sure the boot
        archives are up to date, or behavior is undefined.

    1.3. Fast reboot with unix arguments

        # reboot -f -- '/platform/i86pc/kernel/amd64/unix'
        # reboot -f -- '/platform/i86pc/kernel/unix'

        # reboot -f -- '-k'
        # reboot -f -- '-kv'

        # reboot -f -- '/platform/i86pc/kernel/amd64/unix -k'
        # reboot -f -- '/platform/i86pc/kernel/unix -k'

        # reboot -f -- '/platform/i86pc/my-kernel/amd64/unix -k'
        # reboot -f -- '/platform/i86pc/my-kernel/unix -k'


    1.4. Fast reboot with root disk or pool arguments

        # reboot -f -- '/dev/dsk/ctd0s0'
        # reboot -f -- '/dev/dsk/ctd0s0 /platform/i86pc/kernel/amd64/unix'
        # reboot -f -- '/dev/dsk/ctd0s0 /platform/i86pc/kernel/amd64/unix -k'

       Or with root pool:

        # reboot -f -- 'rpool/ROOT/zfsbe1'
        # reboot -f -- 'rpool/ROOT/zfsbe2 /platform/i86pc/kernel/amd64/unix'
        # reboot -f -- 'rpool/ROOT/zfsbe3 /platform/i86pc/kernel/amd64/unix -k'

    1.5. Fast reboot with -e option

        # reboot -f -e s3
        # reboot -f -e s3 -- '/platform/i86pc/kernel/amd64/unix'
        # reboot -f -e s3 -- '-k'
        # reboot -f -e s3 -- '/platform/i86pc/kernel/amd64/unix -k'

    1.6. Fast reboot with alternate mounted root file system

        # reboot -f -- '/mnt/platform/i86pc/kernel/amd64/unix'
        # reboot -f -- '/mnt/platform/i86pc/kernel/amd64/unix -k'


2. Known-to-work Drivers

    2.1 NIC Drivers

        e1000g
        bge
        nge
        xge
        igb
        ixgbe
        rge
        chxge

    2.2 HBA Drivers

        nv_sata
        marvell88sx
        mpt
        aac

    2.3 USB Drivers

        ehci
        ohci
        uhci

    2.4 Misc Drivers

        bscv
        fdc
        mscsi
        ioat
        nvidia (will be in build 100)

3. Known-to-work Platforms

    Fast reboot works on the following platforms when they use devices
    with known-to-work drivers.

    3.1 Sun Fire Servers
        x2100, x2200 and their M2 counterparts
        x4100, x4200, x4500, x4600 and their M2 counterparts
        v20z, v40z
        v65x

    3.2 Sun Blade Servers
        x6220, x6250
        x8400

    3.3 Desktops
        Ultra20, Ultra40 and their M2 counterparts.
        Ultra24

        NOTE: nvidia currently does not have quiesce(9E)
        implementation.  If nvidia is the only driver shown by "reboot
        -f dryrun" as not having a quiesce(), user can safely force
        fast reboot by putting

                set force_fastreboot = 1

        in /etc/system, or

                # echo "force_fastreboot/W 1" | mdb -kw

    3.4 Other

        Intel Clovertown-based whitebox


4. How To Test Your quiesce(9E)

    - Install build 100 or above
    - bfu to the latest bits
    - Run the following command:
        # reboot -f dryrun
        # grep "no quiesce" /var/adm/messages
      Hopefully your driver is the only one without quiesce().  If not,
      file bugs against drivers without quiesce().
    - Install your driver, or install your kernel with the new driver.
    - Run the following command
        # reboot -f

      Between "rebooting..." to the Solaris should take less than 5
      seconds.  If the system hangs or goes through BIOS, the
      implmenetation is no good.

    - If the first "reboot -f" works
        * Boot kmdb
        * Put the following script in /etc/rc3.d/
      then type "reboot -f".  This will run fast reboot in a loop.  If
      the system evern drops into kmdb prompt or hangs or resets, the
      implementation is no good.

        #!/bin/sh
        LOOPFILE=/var/tmp/loops
        if [ -f $LOOPFILE ]; then
            loops=`cat /var/tmp/loops`;
        else
            loops=0;
        fi
        echo $loops
        loops=`expr $loops + 1`
        echo $loops > /var/tmp/loops
        sleeptime=`expr $loops % 20`
        sleep $sleeptime

        /usr/sbin/reboot -f


5. Known Issues

    5.1 ON bugs you might run into:

        6703055 assertion failed: (curcpup())->cpu_flags & 0x002
        6709808 assertion failed in bio.c during reboot
        6747441 GRUB/vdev_get_bootpath() and zpool_get_physpath() should
                be recursive

    5.2 Possible conditions where fast reboot would fail

        5.2.1 Not all drivers have implemented quiesce(9E)

        Sep 18 13:19:12 too-cool genunix: WARNING: nvidia has no quiesce()
        reboot: not all drivers have implemented quiesce(9E)

        If you NIC and display are the only drivers that don't support
        quiesce(9E), you can unplumb the interface first then force
        fast reboot.  Like this:

        # ifconfig iwh0 unplumb
        # ifconfig iwh0 inet6 unplumb (if you have configured inet6)
        # modunload -i 0
        # echo "force_fastreboot/W 1" | mdb -kw
        # reboot -f

        If you are happy with the result, ie, it works, you can put the
        following line in /etc/system:
        # echo "set force_fastreboot=1" >> /etc/system
        
        If you still need to following steps 1-3 before typing "reboot -f".

        5.2.2 Not enough memory

        If there is not enough memory below 1G (0x40000000) for
        building the page tables, or not enough free memory to load the
        new kernel and boot archive, the fast reboot attempt will fail
        with such messages and fall back to regular reboot:

        Fastboot: Couldn't allocate size below PA 1G to do fast reboot
        Fastboot: Couldn't allocate size below PA 64G to do fast reboot

        5.2.3 Not supported

        Fast reboot is currently not supported in xVM or zones.


6. Features (Not Bugs)

    6.1 Transient menu entry

        When rebooting to a different root disk or pool via mountpoint
        or boot environment, no transient menu entry will be added to
        menu.lst.

    6.2 Limited functionality for "uadmin 2 8"

        Most of the options available for "reboot -f" is not present
        for "uadmin 2 8".  This is a feature, not a bug.  We want to
        discourage the use of uadmin(1M) as the reboot command due to
        two main limitations:

            - uadmin(1M) doesn't update the boot archive
            - uadmin(1M) doesn't update the menu

7. A little secret

    If you know that you system is capable of fast reboot, you can do
    the following to make fast reboot the default behavior of
    reboot(1M).

            # touch /etc/fastreboot