Arrowaver Row's memoir

Moving Solaris 10 update 4 VM from VMware into VirtualBox

I wanted to release my Sol10u4 from VMware Player 2.0.2 environment and move it into SUN VirtualBox (3.0.8). There are some really good features favoring Virtualbox when comparing these two free products.

 But, yes, there could be some glitches. I've spent some time solving them, but in reward the VM was working fine.

Here are steps I went through:

1. shutdown OS in VMware player correctly

2. copy the VM files into new location

3. define entry for existing virtual HDD in VirtualBox. I set first file of set .vmdk so VBox  gets all the rest (in same dir) by itself. For ATA HDD I set PII4 and no additional controllers

4. try to boot

5. if you encounter panic try to find root cause, It was unable to mount root in my case. You can find hard to catch the panic message and be unable to pause the VM, because it will just reboot when panicing. Try printscreen of your desktop to get it if needed

6. you should be still able to boot Failsafe from GRUB menu, let it mount your OS instance under /a as read-write FS

7. identify boot disk by running format utility, confirm it is IDE(c0d0) or e.g. AHCI/SATA(SCSI-like, c0t0d0) and fix in VBox configuration if it was set to the other type

8. Failsafe session should identify your OS instance and offer you to mount it e.g. /a, check bootpath entry in /a/boot/solaris/bootenv.rc, it should point to the bootdisk found in format, correct the entry otherwise fix it with vi editor (you would probably need to set shell variables e.g. TERM=vt100, EDITOR=vi first and than export them)

9. correct /a/etc/vfstab if needed

10. now you need to correct links in /a/dev/dsk and /a/dev/rdsk so they point into right device files in /a/devices

I believe this step really depends on your situation, in my case remaning /pci@0,0/pci-ide@7,1 to  /pci@0,0/pci-ide@1,1 and removing c0* in /a/dev/[r]dsk and than running devfsadm -R /a was OK. But you can end up with creating device nod and its links manually.

11. you will need to fix you network config now or later when OS boots fine. I have Intel PRO/1000 MT Desktop defined as the only NIC in VM, so renamed hostname.pcn0 and dhcp.pcn0 both to .e1000g0.

12. rename VMware tools startup scripts in /a/etc/rc2.d if found any to disable it or detele it.

13. You could have problem run X or your screen is blank after boot, to overcome this you need to remove /etc/X11/xorg.conf now (the VMware Tools created one, VBox's Guest addtions will create new after its installation)

14. run "bootadm update-archive -R /a" to get updated bootarchive. Do touch /a/reconfigure file just for sure and cd / ; umount /a than reboot.

15. If you fail with proper defining your dev and devices you can end up on failing svc:/system/filesystem/xyz and your root will be mounted on device file

16. When booted&logged into X set front panel network applet (animated icon) NIC device from pcn0 to e1000g via its properties if using Gnome desktop

17. Do graceful shutdown

18. Convert vmdk format into VDI using VBoxMange clonehd

e.g.

C:\Program Files\Sun\VirtualBox>VBoxManage clonehd "H:\VBoxVMs\MySu4\Solaris 10
update 3.vmdk" H:\VBoxVMs\MySu4-Conv\MySu4-Conv.vdi --format VDI
VirtualBox Command Line Management Interface Version 3.0.8
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.

0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Clone hard disk created in format 'VDI'. UUID: 64fe776e-811e-4b27-afa1-ad9393275
ed8

than you need to define new virtual HDD (.vdi) in Virtual Media Manger and swap it with original .vmdk drive

19. use VBoxManage modifyhd --compact

C:\Program Files\Sun\VirtualBox>VBoxManage modifyhd H:\VBoxVMs\MySu4-Conv\MySu4-
Conv.vdi --compact
VirtualBox Command Line Management Interface Version 3.0.8
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.

0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%

C:\Program Files\Sun\VirtualBox>

20. Boot from new HDD(.vdi)  and install Guest Additions and reboot. You can try mount -F hsfs -o ro /dev/sr0 /mnt if unable to mount VBox built-in Guest additions image into OS. Than cd /mnt ; pkgadd -d ./VBoxSolarisAdditions.pkg.

21. create snapshot of your just-migrated VM, just for sure ;o)

If you have VM/OS stability issues

Be sure to set following for VM configuration:

1. turn off: direct HW virtual support + IOAPIC + PAEX

2. set 2GB of RAM and just 1 CPU

3. turn on: VT-x/AMD-V

4. disable USB & Serial

Posted at 10:48dop. XI 27, 2009 by Arrowaver in Sun  |  Comments[0]

Error boot opensolaris 2008.05: panic Page fault occurred in module zfs due to a NULL pointer dereference or how to move boot file system to other zpool

I definitely recommend you to use opensolaris 2009.06, but if you for some reason stay on 2008.05 you can encounter problem same as I had. For me it came after power outage - and I wanted my system recovered as I use it as virtualization platform (14 quests) on VirtualBox hypervisor.

When booted into single-user from LiveCD I saw my booting ZFS pool and root file system is OK and consistent. It was not rpool zpool (the zpool name was bootpool) because I did "booting zpool migration" once in the past. So it came in my mind to try fix my system boot problem this way - creating new zpool and migrate booting file system into it. I'd tried to find proper steps in summer 2008, but even did boot filesystem migration successfully - I didn't documented the steps ;o(

The original thread is here: http://www.opensolaris.org/jive/thread.jspa?messageID=248211&#248211

Now my system was panicing in a loop on page fault occurred in module zfs due to a NULL pointer dereference, no matter I removed snapshots, clones, other boot environments (beadm (1M)). I'm not exactly sure of root cause - this happened just after system lost it power. I've never saw this system panic before this one for any reason.

The console output is here:

SunOS Release 5.11 Version snv_86 64-bit
Copyright 1983-2008 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
Hostname: testopen

panic[cpu0]/thread=ffffff04e391f000: BAD TRAP: type=e (#pf Page fault) rp=ffffff001f2e7620 addr=28 occurred in module "zfs" due to a NULL pointer dereference

zfs: #pf Page fault
Bad kernel fault at addr=0x28
pid=80, pc=0xfffffffff78599ec, sp=0xffffff001f2e7710, eflags=0x10282
cr0: 8005003b<pg,wp,ne,et,ts,mp,pe> cr4: 6f8<xmme,fxsr,pge,mce,pae,pse,de>
cr2: 28cr3: 1fffd1000cr8: c

        rdi:                0 rsi:                1 rdx:                2
        rcx:                3  r8:       28004fac00  r9:                1
        rax:                0 rbx:                0 rbp: ffffff001f2e7720
        r10:                1 r11:                1 r12: ffffff04f35e26c0
        r13:                0 r14:           280a0b r15:                0
        fsb:                0 gsb: fffffffffbc259f0  ds:               4b
         es:               4b  fs:                0  gs:              1c3
        trp:                e err:                0 rip: fffffffff78599ec
         cs:               30 rfl:            10282 rsp: ffffff001f2e7710
         ss:               38

ffffff001f2e7500 unix:die+c8 ()
ffffff001f2e7610 unix:trap+13b9 ()
ffffff001f2e7620 unix:cmntrap+e9 ()
ffffff001f2e7720 zfs:vdev_is_dead+c ()
ffffff001f2e7770 zfs:vdev_mirror_child_select+68 ()
ffffff001f2e77b0 zfs:vdev_mirror_io_start+11f ()
ffffff001f2e77f0 zfs:zio_vdev_io_start+156 ()
ffffff001f2e7820 zfs:zio_execute+7f ()
ffffff001f2e7860 zfs:zio_wait+2e ()
ffffff001f2e7910 zfs:arc_read+6e3 ()
ffffff001f2e79d0 zfs:dmu_objset_open_impl+d1 ()
ffffff001f2e7a40 zfs:dsl_pool_open+59 ()
ffffff001f2e7b00 zfs:spa_load+37d ()
ffffff001f2e7b70 zfs:spa_open_common+cc ()
ffffff001f2e7be0 zfs:spa_get_stats+4a ()
ffffff001f2e7c30 zfs:zfs_ioc_pool_stats+32 ()
ffffff001f2e7cb0 zfs:zfsdev_ioctl+10d ()
ffffff001f2e7cf0 genunix:cdev_ioctl+48 ()
ffffff001f2e7d30 specfs:spec_ioctl+86 ()
ffffff001f2e7db0 genunix:fop_ioctl+7b ()
ffffff001f2e7ec0 genunix:ioctl+174 ()
ffffff001f2e7f10 unix:brand_sys_syscall32+197 ()

syncing file systems... done
dumping to /dev/dsk/c10t0d0s1, offset 431030272, content: kernel
100% done: 225221 pages dumped, compression ratio 5.53, dump succeeded
rebooting...
 

When you want just migrate your booting zpool  into another zpool the process is the same as there is nothing specific to my problem - you can consider my system to be operational again as side effect here.

so here you can read the steps which lead back to working system after new booting zpool was created:

1. =>BOOT FROM LIVECD OF THE SAME OPENSOLARIS VERSION TO SINGLE USERMODE

SunOS Release 5.11 Version snv_86 64-bit
Copyright 1983-2008 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
Booting to milestone "milestone/single-user:default".
Hostname: opensolaris
Remounting root read/write
Probing for device nodes ...
Preparing live image for use
Done mounting Live image

USB keyboard
 1. Albanian                      22. Latvian
 2. Belarusian                    23. Macedonian
 3. Belgian                       24. Malta_UK
 4. Bulgarian                     25. Malta_US
 5. Croatian                      26. Norwegian
 6. Czech                         27. Polish
 7. Danish                        28. Portuguese
 8. Dutch                         29. Russian
 9. Finnish                       30. Serbia-And-Montenegro
10. French                        31. Slovenian
11. French-Canadian               32. Slovakian
12. Hungarian                     33. Spanish
13. German                        34. Swedish
14. Greek                         35. Swiss-French
15. Icelandic                     36. Swiss-German
16. Italian                       37. Traditional-Chinese
17. Japanese-type6                38. TurkishQ
18. Japanese                      39. TurkishF
19. Korean                        40. UK-English
20. Latin-American                41. US-English
21. Lithuanian
To select the keyboard layout, enter a number [default 41]:
1. Chinese - Simplified
2. Chinese - Traditional
3. English
4. French
5. German
6. Italian
7. Japanese
8. Korean
9. Portuguese - Brazil
10. Russian
11. Spanish
12. Swedish
To select the desktop language, enter a number [default 3]:

Configuring devices.
Mounting local partitions/cdroms
Using physical swap on hdd (/dev/dsk/c10t0d0s1)
Requesting System Maintenance Mode
SINGLE USER MODE
Root password for system maintenance (control-d to bypass):
single-user privilege assigned to /dev/console.
Entering System Maintenance Mode
PASSWORD IS "opensolaris"
 

2. =>IMPORT ORIGINAL BOOT ZPOOL WITH -f

May 26 09:39:16 su: 'su root' succeeded for root on /dev/console
zpool imSun Microsystems Inc.   SunOS 5.11      snv_86  January 2008
port bo-bash-3.2# zpool import bootpool
cannot import 'bootpool': pool may be in use from other system
use '-f' to import anyway
-bash-3.2# zpool import -f bootpool

3. =>CREATE OR IMPORT OTHER ZPOOL (SAME PHYS HDD TYPE SHOULD BE PREFERRED) WHICH RESIDES ON SOLARIS2 FDISK PARTITION AND HAS SLICE DEFINED

-bash-3.2# zpool import bootpool2
cannot import 'bootpool2': pool may be in use from other system
use '-f' to import anyway
-bash-3.2# zpool import -f bootpool2
-bash-3.2#
-bash-3.2# zpool list
NAME        SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
bootpool    928G  90.1G   838G     9%  ONLINE  -
bootpool2   928G  2.39M   928G     0%  ONLINE  -

This could be second disk in ZFS mirror, this mean you have to detach this disk from zpool first and force -f the zpool creation.

4. =>CREATE ZFS SNAPSHOTS FOR YOUR ORIGINAL FSs HERE NAMED AS @2

-bash-3.2# zfs list
NAME                                                        USED  AVAIL  REFER  MOUNTPOINT
bootpool                                                   90.1G   823G  56.5K  /bootpool
bootpool@2                                                   19K      -  56.5K  -
bootpool/ROOT                                              90.1G   823G    18K  /bootpool/ROOT
bootpool/ROOT@2                                              15K      -    18K  -
bootpool/ROOT/opensolaris                                  90.1G   823G  65.2G  legacy
bootpool/ROOT/opensolaris@Mar032009befVirtBOXupgradeTo214  24.8G      -  66.8G  -
bootpool/ROOT/opensolaris@send2backup                        68K      -  65.2G  -
bootpool/ROOT/opensolaris@2                                    0      -  65.2G  -
bootpool2                                                  2.29M   913G  56.5K  /bootpool2
bootpool2@2                                                    0      -  56.5K  -
bootpool2/ROOT                                               18K   913G    18K  /bootpool2/ROOT
bootpool2/ROOT@2                                               0      -    18K  -
-bash-3.2#
<l/ROOT/opensolaris@2 | zfs receive bootpool2/ROOT/opensolaris

5. =>SEND THE SNAPHOST's DATA INTO NEW ZPOOL CREATED EARLIER

zfs send bootpool@2 | zfs receive -F bootpool2
zfs send bootpool/ROOT@2 | zfs receive bootpool2/ROOT
zfs send bootpool/ROOT/opensolaris@2 | zfs receive bootpool2/ROOT/opensolaris

This will sent all file systems' data into new zpool created in step 3.

6. =>SET NEW ROOT FS MOUNT POINT AS LEGACY

-bash-3.2# zfs get mountpoint bootpool/ROOT/opensolaris
NAME                       PROPERTY    VALUE                      SOURCE
bootpool/ROOT/opensolaris  mountpoint  legacy                     local
-bash-3.2# zfs get mountpoint bootpool2/ROOT/opensolaris
NAME                        PROPERTY    VALUE                        SOURCE
bootpool2/ROOT/opensolaris  mountpoint  /bootpool2/ROOT/opensolaris  default
-bash-3.2#
-bash-3.2# zfs set mountpoint=legacy bootpool2/ROOT/opensolaris
-bash-3.2# zfs get mountpoint bootpool2/ROOT/opensolaris
NAME                        PROPERTY    VALUE                       SOURCE
bootpool2/ROOT/opensolaris  mountpoint  legacy                      local
-bash-3.2#

This need to be done for proper boot process and to be referenced in /etc/vfstab see step 8.

7. =>MOUNT THE NEW ROOT

mount -F zfs bootpool2/ROOT/opensolaris /mnt

8. =>EDIT MENU.LST AND VFSTAB IN NEW ZPOOL TO REFLECT ZPOOL NAME CHANGE

 -bash-3.2# cat /bootpool2/boot/grub/menu.lst
serial --unit=0 --speed=9600 --word=8 --parity=no --stop=1
terminal serial
##splashimage /boot/grub/splash.xpm.gz
timeout 30
default 0
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title OpenSolaris 2008.05 snv_86_rc3 X86 IEDIed
bootfs bootpool2/ROOT/opensolaris
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS,console=ttya
module$ /platform/i86pc/$ISADIR/boot_archive
#---------------------END BOOTADM--------------------

# Unknown partition of type 238 found on /dev/rdsk/c10t1d0p0 partition: 1
# It maps to the GRUB device: (hd0,0) .

# Unknown partition of type 238 found on /dev/rdsk/c10t4d0p0 partition: 1
# It maps to the GRUB device: (hd1,0) .
title bootCLONE
bootfs bootpool/ROOT/bootCLONE
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS
module$ /platform/i86pc/$ISADIR/boot_archive
#============ End of LIBBE entry =============
-bash-3.2#
-bash-3.2# cat /mnt/etc/vfstab
#device         device          mount           FS      fsck    mount   mount
#to mount       to fsck         point           type    pass    at boot options
#
/devices        -       /devices        devfs   -       no      -
/proc   -       /proc   proc    -       no      -
ctfs    -       /system/contract        ctfs    -       no      -
objfs   -       /system/object  objfs   -       no      -
sharefs -       /etc/dfs/sharetab       sharefs -       no      -
fd      -       /dev/fd fd      -       no      -
swap    -       /tmp    tmpfs   -       yes     -
bootpool2/ROOT/opensolaris      -       /       zfs     -       no      -
/dev/dsk/c10t0d0s1      -       -       swap    -       no      -
/dev/dsk/c10t4d0s1      -       -       swap    -       no      -
-bash-3.2#

9. =>EXPORT ORIGINAL BOOT ZPOOL AND LEAVE ONLY NEW ZPOOL IMPORTED THAT COPY ZPOOL.CACHE TO NEW ROOT

-bash-3.2# zpool list
NAME        SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
bootpool    928G  90.1G   838G     9%  ONLINE  -
bootpool2   928G  65.2G   863G     7%  ONLINE  -
-bash-3.2# ls -l /etc/zfs/zpool.cache
-rw-r--r--   1 root     root        2084 May 26 09:41 /etc/zfs/zpool.cache
-bash-3.2# zpool export bootpool
-bash-3.2# ls -l /etc/zfs/zpool.cache
-rw-r--r--   1 root     root        1040 May 27 01:29 /etc/zfs/zpool.cache
-bash-3.2# ls -l /mnt/etc/zfs
total 52
-rw-r--r--   1 root     root       13140 Apr 10 06:19 _no_zpool.cache
-rw-r--r--   1 root     root       11988 May 15 04:52 zpool.cache
-bash-3.2# mv /mnt/etc/zfs/zpool.cache /mnt/etc/zfs/_no2_zpool.cache
-bash-3.2# ls -l /mnt/etc/zfs
total 52
-rw-r--r--   1 root     root       11988 May 15 04:52 _no2_zpool.cache
-rw-r--r--   1 root     root       13140 Apr 10 06:19 _no_zpool.cache
-bash-3.2#
-bash-3.2#
-bash-3.2# cp /etc/zfs/zpool.cache /mnt/etc/zfs/
-bash-3.2# ls -l /mnt/etc/zfs
total 56
-rw-r--r--   1 root     root       11988 May 15 04:52 _no2_zpool.cache
-rw-r--r--   1 root     root       13140 Apr 10 06:19 _no_zpool.cache
-rw-r--r--   1 root     root        1040 May 27 01:36 zpool.cache
-bash-3.2#

Copy your zpool.cache (when still booted form LiveCD) in new zpool root file system.

10. =>SET BOOTFS FOR THE NEW ZPOOL

-bash-3.2# zpool get bootfs bootpool2
NAME       PROPERTY  VALUE      SOURCE
bootpool2  bootfs    -          default
-bash-3.2#
-bash-3.2# zpool set bootfs=bootpool2/ROOT/opensolaris bootpool2
-bash-3.2# zpool get bootfs bootpool2
NAME       PROPERTY  VALUE                       SOURCE
bootpool2  bootfs    bootpool2/ROOT/opensolaris  local
-bash-3.2#

11. =>UPDATE BOOT ARCHIVE ON NEW ROOT

-bash-3.2#
-bash-3.2# bootadm update-archive -v -R /mnt
cannot find: /mnt/etc/cluster/nodeid: No such file or directory
cannot find: /mnt/etc/devices/mdi_ib_cache: No such file or directory
cannot find: /mnt/etc/devices/retire_store: No such file or directory
cannot find: /mnt/platform/i86pc/ucode/AuthenticAMD: No such file or directory
    changed /mnt/etc/zfs/zpool.cache
Creating boot_archive for /mnt
updating /mnt/platform/i86pc/amd64/boot_archive
updating /mnt/platform/i86pc/boot_archive
-bash-3.2#
-bash-3.2#

12. => UMOUNT ROOT

-bash-3.2# umount /mnt

13. =>NOW BOOT FROM NEW ROOT - CHECK SYSTEM BIOS TO TARGET RIGTH DISK FOR BOOTING

-bash-3.2# reboot
syncing file systems... done
rebooting...
S

    GNU GRUB  version 0.95  (619K lower / 4094912K upper memory)

 +-------------------------------------------------------------------------+
 | OpenSolaris 2008.05 snv_86_rc3 X86 IEDIed                               |
 | bootCLONE                                                               |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 +-------------------------------------------------------------------------+
      Use the ^ and v keys to select which entry is highlighted.
      Press enter to boot the selected OS, 'e' to edit the
      commands before booting, or 'c' for a command-line.

    GNU GRUB  version 0.95  (619K lower / 4094912K upper memory)

 +-------------------------------------------------------------------------+
 | bootfs bootpool2/ROOT/opensolaris                                       |
 | kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS,console=tt>  |
 | module$ /platform/i86pc/$ISADIR/boot_archive                            |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 +-------------------------------------------------------------------------+
      Use the ^ and v keys to select which entry is highlighted.
      Press 'b' to boot, 'e' to edit the selected command in the
      boot sequence, 'c' for a command-line, 'o' to open a new line
      after ('O' for before) the selected line, 'd' to remove the
      selected line, or escape to go back to the main menu.
SunOS Release 5.11 Version snv_86 64-bit
Copyright 1983-2008 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
Hostname: testopen
Reading ZFS config: done.
Mounting ZFS filesystems: (3/3)
May 27 09:26:39 testopen smbsrv: NOTICE: SmbOplocks: disabled
May 27 09:26:44 testopen smbd[409]: Failed to load share share2
May 27 09:26:44 testopen smbd[409]: Failed to load share share3
May 27 09:26:44 testopen smbd[409]: Failed to load share sharerow
May 27 09:26:44 testopen smbd[409]: Failed to load share share1

testopen console login: row
Password:
No directory! Logging in with home=/
Last login: Mon May 11 11:50:19 from sr1-eprg06-07.c
Sun Microsystems Inc.   SunOS 5.11      snv_86  January 2008
-bash-3.2$ su -
Password:
May 27 09:35:38 testopen last message repeated 2 times
May 27 09:35:55 testopen su: 'su root' succeeded for row on /dev/console
Sun Microsystems Inc.   SunOS 5.11      snv_86  January 2008
-bash-3.2#
-bash-3.2#
-bash-3.2#
-bash-3.2# zpool list
NAME        SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
bootpool2   928G  66.2G   862G     7%  ONLINE  -
-bash-3.2# zpool status
  pool: bootpool2
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        bootpool2   ONLINE       0     0     0
          c8t4d0s0  ONLINE       0     0     0

errors: No known data errors
-bash-3.2# df -h
Filesystem             size   used  avail capacity  Mounted on
bootpool2/ROOT/opensolaris
                       914G    66G   847G     8%    /
/devices                 0K     0K     0K     0%    /devices
/dev                     0K     0K     0K     0%    /dev
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                    16G   836K    16G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
sharefs                  0K     0K     0K     0%    /etc/dfs/sharetab
/usr/lib/libc/libc_hwcap2.so.1
                       913G    66G   847G     8%    /lib/libc.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                    16G    16K    16G     1%    /tmp
swap                    16G    56K    16G     1%    /var/run
bootpool2              914G    56K   847G     1%    /bootpool2
bootpool2/ROOT         914G    18K   847G     1%    /bootpool2/ROOT
-bash-3.2#

Now you can try to boot from new zpool - in GRUB you can type "e" - edit to see if your booting zpool change is in place (the menu.lst file). Be careful the directive "bootfs" is specific to same releases of opensolaris, new versions use "findroot" directive.

Zpool cache file replacement caused all other zpools not to be imported this time - you need to do it manualy just once in the next step to make it automatic during boot.

14. =>RUN ZPOOL IMPORT TO FIND&IDENTIFY ALL OTHER ZPOOLs YOU NEED AND IMPORT THEM (THEY WILL BE IMPORTED AUTOMATICATILY NEXT TIME - ZPOOL.CACHE)

-bash-3.2#
-bash-3.2# zpool import
  pool: VBOXvms1
    id: 1020763366063509716
 state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:

        VBOXvms1     ONLINE
          mirror     ONLINE
            c11t0d0  ONLINE
            c4t0d0   ONLINE

  pool: bootpoolBackup
    id: 15922132069583834627
 state: UNAVAIL
status: The pool is formatted using an incompatible version.
action: The pool cannot be imported.  Access the pool on a system running newer
        software, or recreate the pool from backup.
   see: http://www.sun.com/msg/ZFS-8000-A5
config:

        bootpoolBackup  UNAVAIL  newer version
          c12t7d0   ONLINE

  pool: bootpool
    id: 2411838337529240089
 state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:

        bootpool    ONLINE
          c8t0d0s0  ONLINE

  pool: CIFSshare2
    id: 14937498107472144881
 state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:

        CIFSshare2   ONLINE
          mirror     ONLINE
            c12t1d0  ONLINE
            c12t3d0  ONLINE
          mirror     ONLINE
            c9t5d0   ONLINE
            c9t6d0   ONLINE

  pool: CIFSshare
    id: 6146690073225590487
 state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:

        CIFSshare    ONLINE
          mirror     ONLINE
            c12t2d0  ONLINE
	    c9t4d0   ONLINE
          mirror     ONLINE
            c9t2d0   ONLINE
            c12t4d0  ONLINE

  pool: rpool
    id: 10610396790796762733
 state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:

        rpool       ONLINE
          mirror    ONLINE
            c5t0d0  ONLINE
            c9t3d0  ONLINE
          mirror    ONLINE
            c5t1d0  ONLINE
            c5t2d0  ONLINE
          mirror    ONLINE
            c9t1d0  ONLINE
            c5t3d0  ONLINE

-bash-3.2#
-bash-3.2# zpool import VBOXvms1
-bash-3.2# zpool import CIFSshare2
-bash-3.2# zpool import CIFSshare
-bash-3.2# zpool import rpool
-bash-3.2#

Import all zpools you want to be imported automatically during boot process.

15. =>IMPORT ORGINAL POOL AND DESTROY IT

-bash-3.2#
-bash-3.2# zpool import bootpool
-bash-3.2# zpool destroy bootpool

He is time to destroy original booting zpool, which cannot be used for boot anyway and you probably want to release the disk for mirroring in new zpool created in step 3.

16. =>ATTACH SECOND DISK TO THE BOOTING ONE (YOU SHOULD BE ABLE TO DO WITHOUT  -f)

-bash-3.2# zpool status bootpool2
  pool: bootpool2
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        bootpool2   ONLINE       0     0     0
          c8t4d0s0  ONLINE       0     0     0

errors: No known data errors

-bash-3.2# zpool attach -f bootpool2 c8t4d0s0 c8t0d0s0
-bash-3.2# zpool status bootpool2
  pool: bootpool2
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
 scrub: resilver in progress for 0h0m, 0.51% done, 0h22m to go
config:

        NAME          STATE     READ WRITE CKSUM
        bootpool2     ONLINE       0     0     0
          mirror      ONLINE       0     0     0
            c8t4d0s0  ONLINE       0     0     0
            c8t0d0s0  ONLINE       0     0     0

errors: No known data errors
-bash-3.2# zpool status bootpool2
  pool: bootpool2
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
 scrub: resilver in progress for 0h1m, 5.69% done, 0h27m to go
config:

        NAME          STATE     READ WRITE CKSUM
        bootpool2     ONLINE       0     0     0
          mirror      ONLINE       0     0     0
            c8t4d0s0  ONLINE       0     0     0
            c8t0d0s0  ONLINE       0     0     0

errors: No known data errors
-bash-3.2#

Whenever you need -f or not depends on attached disk history.

17. =>AFTER RESILVER COMPLETES YOU CAN REBOOT TO SEE EVERYTHING OK

-bash-3.2# zpool status bootpool2
  pool: bootpool2
 state: ONLINE
 scrub: none requested
config:

        NAME          STATE     READ WRITE CKSUM
        bootpool2     ONLINE       0     0     0
          mirror      ONLINE       0     0     0
            c8t4d0s0  ONLINE       0     0     0
            c8t0d0s0  ONLINE       0     0     0

errors: No known data errors
-bash-3.2#
-bash-3.2# fdisk c8t0d0p0
             Total disk size is 60800 cylinders
             Cylinder size is 32130 (512 byte) blocks

                                               Cylinders
      Partition   Status    Type          Start   End   Length    %
      =========   ======    ============  =====   ===   ======   ===
          1       Active    Solaris2          1  60799    60799    100



SELECT ONE OF THE FOLLOWING:
   1. Create a partition
   2. Specify the active partition
   3. Delete a partition
   4. Change between Solaris and Solaris2 Partition IDs
   5. Exit (update disk configuration and exit)
   6. Cancel (exit without updating disk configuration)
Enter Selection: 6

-bash-3.2# fdisk c8t4d0p0

             Total disk size is 60800 cylinders
             Cylinder size is 32130 (512 byte) blocks

                                               Cylinders
      Partition   Status    Type          Start   End   Length    %
      =========   ======    ============  =====   ===   ======   ===
          1       Active    Solaris2          1  60799    60799    100


SELECT ONE OF THE FOLLOWING:
   1. Create a partition
   2. Specify the active partition
   3. Delete a partition
   4. Change between Solaris and Solaris2 Partition IDs
   5. Exit (update disk configuration and exit)
   6. Cancel (exit without updating disk configuration)
Enter Selection: 6

-bash-3.2# prtvtoc /dev/rdsk/c8t0d0s2
* /dev/rdsk/c8t0d0s2 partition map
*
* Dimensions:
*     512 bytes/sector
*     126 sectors/track
*     255 tracks/cylinder
*   32130 sectors/cylinder
*   60799 cylinders
*   60797 accessible cylinders
*
* Flags:
*   1: unmountable
*  10: read-only
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       0      2    00    4241160 1949166450 1953407609
       1      3    01      32130   4209030   4241159
       2      5    01          0 1953407610 1953407609
       8      1    01          0     32130     32129
-bash-3.2#
-bash-3.2# prtvtoc /dev/rdsk/c8t4d0s2
* /dev/rdsk/c8t4d0s2 partition map
*
* Dimensions:
*     512 bytes/sector
*     126 sectors/track
*     255 tracks/cylinder
*   32130 sectors/cylinder
*   60799 cylinders
*   60797 accessible cylinders
*
* Flags:
*   1: unmountable
*  10: read-only
*
*                          First     Sector    Last
* Partition  Tag  Flags    Sector     Count    Sector  Mount Directory
       0      2    00    4241160 1949166450 1953407609
       1      3    01      32130   4209030   4241159
       2      5    01          0 1953407610 1953407609
       8      1    01          0     32130     32129
-bash-3.2#
-bash-3.2#
PARTITION TABLES AND VTOCs ARE HERE JUST FOR REFERENCE PURPOSE. 
 

18. =>IF YOU ARE RECOVERING YOU ODLER BOOT DISKS THE GRUB SHOULD BE THERE ON BOTH DISKS OTHERWISE USE INSTALLGRUB

Posted at 04:34odp. VI 02, 2009 by Arrowaver in Sun  |  Comments[0]

JET on OpenSolaris

I've found laptop replacement again a lesson for a few reasons:

• new one came whit Vista which needed to shink NTFS partition first
• next Solaris 10 after installation prevents Vista to boot anymore - some MBR/early boot phase  hacks are available, see http://blogs.sun.com/moinakg/entry/solaris_vista_dual_boot_revisited for more info
  
•  I was testing OpenSolaris2008.05 on x64 server I've found it very pleasant environment. The Vista dual boot problem is solved in snv_82 and OpelSolaris2008.05 is snv_86 - so it was clear choice.
• unfortunately it could not recognize my NIC and soma other not very important peripherals
• my bright colleague point me to opensolaris.org for osol-0811-99.iso
  
• there is RC of 2008.11 now
• with snv_99 laptop is running fine (Vista boot OK, added to GRUB during install smoothly)

Now it was time to setup JET, which is easy task for standard environment when you doing it at least every few months not years :o)

JET SETUP STEPs

1. Install JET packages

I've installed following packages, here you can rely on OpenSolaris backward compatibility with Solaris packaging system. Don't care much if some packages are displayed as for Sparc platform and continue.

application JetEISCD                        EIS CD JET module
application JetEXPLO                        jet explo product
application JetFLASH                        JET flash product
application JetISO                          JET ISO product
application JetSDS                          JET sds product
application JetVTS                          JET VTS product
application SUNWjet                         Sun JumpStart Enterprise Toolkit

Version of packages can be found HERE.

2. Create directory where will stage ISOs. You can use fat32 for sharing ISO with Windows, but be aware of 4GB limit. I prefer separate dataset on extra zpool.

STAGE                 9.5G   20K  9.5G   1% /STAGE
STAGE/Img              40G   30G  9.5G  76% /STAGE/Img

root@testopen3:/Inst/Prog/JET# zpool status
  pool: STAGE
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        STAGE       ONLINE       0     0     0
          c3t0d0p4  ONLINE       0     0     0

errors: No known data errors

  pool: rpool
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        rpool       ONLINE       0     0     0
          c3t0d0s0  ONLINE       0     0     0

errors: No known data errors
root@testopen3:/Inst/Prog/JET#

It was fat32 "OS sharing" partition, but I've found 4GB file limit too limiting.

3. Prepare&copy your ISOs. You can use readcd(1) to create image form you CD/DVD drive. (readcd -f=/<dir>/<file-name.iso> worked for me)

 root@testopen3:/Inst/Prog/JET# cd /STAGE/Img/
root@testopen3:/STAGE/Img# ls -l
total 15770108
-rw-r--r-- 1 root root  8244133888 Oct 29 14:11 EIS-DVD-INT_Sep2008.iso
drwxrwxrwx 2 root root           7 Nov  8 01:41 FLAR
drwxrwxrwx 2 root root           3 Oct 31 07:18 HB
-rwxrwxrwx 1 root root  2922250240 Oct 23 00:54 SOL_10_508_SPARC.iso
-rw-r--r-- 1 row  staff 2661416960 Nov  4 20:50 sol-10-u6-ga1-sparc-dvd.iso
-rw-r--r-- 1 row  staff 2308833280 Nov  4 20:27 sol-10-u6-ga1-x86-dvd.iso
root@testopen3:/STAGE/Img#

4. Populate JET image location tables

root@testopen3:/opt/SUNWjet/etc# cat solaris_iso_locations
# This file is used to specify the mountpoints for iso images. You need
# to create the mountpoint, update this file, and then run /opt/SUNWjet/bin/
# share_isos. This will lofimount all isos listed here. You will then
# need to run add_solaris_location for each image as normal.
# You should run the share_isos command everytime the JET server is
# rebooted.
#
# Format is:
#
# <location of .iso file>       <mountpoint>
#
# e.g. /export/iso/s10_0508s.iso        /export/install/media/Solaris_10_0508S

/STAGE/Img/SOL_10_508_SPARC.iso         /export/install/media/Solaris_10_0508S
/STAGE/Img/EIS-DVD-INT_Sep2008.iso      /export/install/EIS/Sep08
/STAGE/Img/sol-10-u6-ga1-sparc-dvd.iso  /export/install/media/Solaris_10_1008S
/STAGE/Img/sol-10-u6-ga1-x86-dvd.iso    /export/install/media/Solaris_10_1008X
root@testopen3:/opt/SUNWjet/etc#
root@testopen3:/opt/SUNWjet/etc#
root@testopen3:/opt/SUNWjet/etc# cat solaris_media_locations
SOL_10_0508S    /export/install/media/Solaris_10_0508S/Solaris_10/Tools/add_install_client      /export/install/media/Solaris_10_0508S
SOL_10_1008S    /export/install/media/Solaris_10_1008S/Solaris_10/Tools/add_install_client      /export/install/media/Solaris_10_1008S
SOL_10_1008X    /export/install/media/Solaris_10_1008X/Solaris_10/Tools/add_install_client      /export/install/media/Solaris_10_1008X
root@testopen3:/opt/SUNWjet/etc#

For solaris_media_location you can use add_solaris_location in /opt/SUNWjet/bin.

You can write script like this to have ISOs shared on boot

root@testopen3:/opt/SUNWjet/etc# cat /etc/rc3.d/S20JETshare_isos

/opt/SUNWjet/bin/share_isos

root@testopen3:/opt/SUNWjet/etc#

 After run of share_isos you should have FSs mounted:

root@testopen3:/opt/SUNWjet/etc# df -h

Filesystem            Size  Used Avail Use% Mounted on

rpool/ROOT/opensolaris

                       29G  6.9G   22G  25% /

swap                  1.8G  340K  1.8G   1% /etc/svc/volatile

/usr/lib/libc/libc_hwcap1.so.1

                       29G  6.9G   22G  25% /lib/libc.so.1

swap                  1.8G  304K  1.8G   1% /tmp

swap                  1.8G   44K  1.8G   1% /var/run

STAGE                 9.5G   20K  9.5G   1% /STAGE

STAGE/Img              40G   30G  9.5G  76% /STAGE/Img

rpool/export           29G  6.9G   22G  25% /export

rpool/export/home      22G   19K   22G   1% /export/home

rpool/export/home/row

                       22G  119M   22G   1% /export/home/row

rpool                  22G   49K   22G   1% /rpool

/dev/lofi/1           2.6G  2.6G     0 100% /export/install/media/Solaris_10_0508S

/dev/lofi/2           7.7G  7.7G     0 100% /export/install/EIS/Sep08

/dev/lofi/3           2.5G  2.5G     0 100% /export/install/media/Solaris_10_1008S

/dev/lofi/4           2.2G  2.2G     0 100% /export/install/media/Solaris_10_1008X

/export/install/media/Solaris_10_1008X/boot

                      2.2G  2.2G     0 100% /tftpboot/I86PC.Solaris_10-1

root@testopen3:/opt/SUNWjet/etc#

5. Copy EIS to JET

 He you need first hack. OpenSolaris doesn't know "-n" option of df command, which would fail the process.

HACK: /opt/SUNWjet/bin/copy_eiscd

Find  commended line and replace witch the next one

## Commented out as OpenSol do not know -n of df #IMAGE_TYPE=`df -n ${IMAGE_LOC} | awk -F" " '{print $NF}'`

IMAGE_TYPE=`df -T ${IMAGE_LOC} | tail -1 | awk -F" " '{print $(NF-5)}'`

 Than use copy_eis to copy data from lofi mounted image to standard location (using staged image is much faster then copiing from real DVD)

6. Download DHCP server via IPS package manager

I like the GUI, but you can do it from cmd. Use just text files no need for binary databases.Populate make_dhcp script with entries for your network and run it.

  root@testopen3:/opt/SUNWjet/Products/base_config/solaris# cat make_dhcp
  #!/usr/bin/ksh
  # 
  # Copyright 2005 Sun Microsystems, Inc.  All rights reserved. 
  # Use is subject to license terms.
  ################################################################################ 
  #
  # Synopsis: PXE boot pre-JUMP setups
  #       if DHCP server is notpresent, then activate it 
  #
  # Change the NETWORk details to suit your own
  NETWORK=192.168.99.0
  NETMASK=255.255.255.0
  ROUTER=192.168.99.1 
  dhcpconfig -D -r SUNWfiles -p /var/tmp
  dhcpconfig -N ${NETWORK} -m ${NETMASK} -t ${ROUTER}
  #
  root@testopen3:/opt/SUNWjet/Products/base_config/solaris# 

6.5 Enable TFTP service

TFPT is needed for early net boot phase, but it is not enabled by default in Solaris 10 or OpenSolaris. It not not even defined in /etc/inet/inetd.conf in OpenSolaris, so you need to add the definiton first.

TFTP CONFIGURATION
===================

root@testopen3:~# inetadm | grep tftp
enabled   online         svc:/network/tftp/udp6:default
root@testopen3:~#



FROM SOLARIS 10 IN.TFTPD(1M) MAN PAGE
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTES
     The tftpd server only acknowledges the transfer size  option
     that  is  sent  with  a read request when the octet transfer
     mode is specified.

     The in.tftpd.1m service is managed by the service management
     facility, smf(5), under the service identifier:

       svc:/network/tftp/udp6:default



     Administrative actions on this service,  such  as  enabling,
     disabling,  or  requesting  restart,  can be performed using
     svcadm(1M). Responsibility  for  initiating  and  restarting
     this  service  is delegated to inetd(1M). Use inetadm(1M) to
     make configuration changes and to view configuration  infor-
     mation for this service. The service's status can be queried
     using the svcs(1) command.

     Unlike most smf(5) services, a manifest for the tftp service
     is not included in the system. To create one and enable this
     service, the administrator should:

         1.   Edit /etc/inet/inetd.conf and  uncomment  the  tftp
              entry.

         2.   Run /usr/sbin/inetconv.


     After you run inetconv,  the  svc:/network/tftp/udp6:default
     service is created and enabled.

For OpenSolaris:
~~~~~~~~~~~~~~~~
YOU NEED TO ADD THIS LINE TO INETD.CONF - THERE IS NONE BY DEFAULT:

tftp    dgram   udp6    wait    root    /usr/sbin/in.tftpd      in.tftpd -s /tftpboot

Than you can run inetconv(1).

7. Fix SDS module

This and next section (8.) describes small changes in modules' scripts needed for their proper function in this platform.

This is needed because snv_99 for some reason handles ksh expressions differently:

SOLARIS 10u4 x64
~~~~~~~~~~~~~~~~
root@arrowaver5 # ksh
root@arrowaver5 # JS_PRODUCT=sds
root@arrowaver5 # sds_product_version=default
root@arrowaver5 # eval echo $\"{${JS_PRODUCT}_product_version}\"
${sds_product_version}
root@arrowaver5 # JS_PRODVER="`eval echo $\"{${JS_PRODUCT}_product_version}\"`"
root@arrowaver5 # echo $JS_PRODVER
default
root@arrowaver5 # uname -i ; uname -a
i86pc
SunOS arrowaver5 5.10 Generic_120012-14 i86pc i386 i86pc
root@arrowaver5 # cat /etc/release
                        Solaris 10 8/07 s10x_u4wos_12b X86
           Copyright 2007 Sun Microsystems, Inc.  All Rights Reserved.
                        Use is subject to license terms.
                            Assembled 16 August 2007
root@arrowaver5 #


SOLARIS 10u5 SPARC
~~~~~~~~~~~~~~~~~~
# ksh
# JS_PRODUCT=sds
# sds_product_version=default
# eval echo $\"{${JS_PRODUCT}_product_version}\"
${sds_product_version}
# JS_PRODVER="`eval echo $\"{${JS_PRODUCT}_product_version}\"`"
# echo $JS_PRODVER
default
# uname -i ; uname -a
SUNW,Ultra-5_10
SunOS testsol10u6s 5.10 Generic_127127-11 sun4u sparc SUNW,Ultra-5_10
# 


OPENSOLARIS 1108_99 X64
~~~~~~~~~~~~~~~~~~~~~~~
root@testopen3:/opt/SUNWjet/Products/sds# ksh
# JS_PRODUCT=sds
# sds_product_version=default
# eval echo $\"{${JS_PRODUCT}_product_version}\"
{sds_product_version}
# JS_PRODVER="`eval echo $\"{${JS_PRODUCT}_product_version}\"`"
# echo $JS_PRODVER
{sds_product_version}
# uname -i ; uname -a
i86pc
SunOS testopen3 5.11 snv_99 i86pc i386 i86pc Solaris
# cat /etc/release
                          OpenSolaris 2008.11 snv_99 X86
           Copyright 2008 Sun Microsystems, Inc.  All Rights Reserved.
                        Use is subject to license terms.
                            Assembled 08 October 2008
#

HACK: /opt/SUNWjet/Products/sds/check_client

Again find ⁞ commended line and replace witch the next one

## Didn't work on snv_b99 OpenSol ## JS_PRODVER="`eval echo $\"{${JS_PRODUCT}_product_version}\"`"
JS_PRODVER="$(eval echo $`echo "${JS_PRODUCT}_product_version" `)"

## Didn't work on snv_b99 OpenSol ##            slicevar=`eval echo $"{base_config_profile_${slice}_size}"`
                slicevar="$(eval echo $`echo "base_config_profile_${slice}_size"`)"

## Didn't work on snv_b99 OpenSol ##            slicevar=`eval echo $"{base_config_profile_disk_${slice}_size}"`
                slicevar="$(eval echo $`echo "base_config_profile_${slice}_size"`)"

8. Fix & Prepare EXPLO module

HACK: /opt/SUNWjet/Products/explo/check_client

same again

 ## Didn't work on snv_99 OpenSol ##JS_PRODVER="`eval echo $\"{${JS_PRODUCT}_product_version}\"`"

JS_PRODVER="$(eval echo $`echo "${JS_PRODUCT}_product_version" `)"

ADD TO: /opt/SUNWjet/Products/explo/package.matrix

line at the end:

5.10:5.13:SUNWexplo SUNWexplu

UNPACK & COPY TO: /export/install/pkgs/explo/5.13/sparc/

SUN Explorer  packages for version 5.13 here:

root@testopen3:/export/install/pkgs/explo/5.13/sparc# ls -la

total 6

drwxr-xr-x 4 root root 4 Nov  8 08:34 .

drwxr-xr-x 3 root root 3 Nov  8 08:31 ..

drwxr-xr-x 4 root root 6 Aug  8 22:09 SUNWexplo

drwxr-xr-x 4 root root 6 Aug  8 22:09 SUNWexplu

root@testopen3:/export/install/pkgs/explo/5.13/sparc#

9. Prepare template for your client

 Use make_template command and hostname of client as agrument. Edit all the sections for modules. Never use symbolic links for templates/client-names.

If you want to use DHCP boot for Sparc machine, add `uname -i` to jumpstat.conf JS_DHCP_VENDOR variable:

root@testopen3:/opt/SUNWjet/Templates# grep VEND /opt/SUNWjet/etc/jumpstart.conf

JS_DHCP_VENDOR="SUNW.Ultra-5_10 SUNW.Ultra-30 SUNW.Sun-Fire-15000 SUNW.Sun-Blade-1500"

root@testopen3:/opt/SUNWjet/Templates#

10. Check you NFS server

 You share command to check what is exported. You should definitely avoid two thinks:

• exporting a directory in the structure of already exported directory and vice versa
• editing of /etc/dfs/dfstab by hand 
  

NFSserver will comment out such entries in dftab and will not share it. Use sharemgr(1) cmd instead. To create entries JET needs if not already done use cmds similar to this:

  sharemgr add-share -s /export/install/patches default 
  sharemgr set -P nfs -p anon=0 -s /export/install/patches default 
  sharemgr set -P nfs -p ro=* -S sys -s /export/install/patches default
  

For some reason you have to share exactly /export/install/patches and /export/install/pkgs, NOT just /export/install to have patches and addition of packages working  in JET. Even .../pkgs not properly shared causing EIS patches not applied.

check with:

root@testopen3:/export/install/pkgs/explo/5.13/sparc# sharemgr show -p 
  default nfs=() 
            /opt/SUNWjet   nfs=(anon="0") nfs:sys=(ro="*") 
            /SHARED        nfs=(anon="0") 
            /STAGE/Img/FLAR        nfs=(anon="0") nfs:sys=(ro="*") 
            /dhcp  nfs=(anon="0") nfs:sys=(ro="*") 
            /export/install/media/Solaris_10_1008X         nfs=(anon="0") nfs:sys=(ro="*") 
            /export/install/media/Solaris_10_0508S         nfs=(anon="0") nfs:sys=(ro="*") 
            /export/install/media/Solaris_10_1008S         nfs=(anon="0") nfs:sys=(ro="*") 
            /export/install/EIS/Sep08      nfs=(anon="0") nfs:sys=(ro="*") 
            /rarp/Solaris_10_0508S/Solaris_10/Tools/Boot   nfs=(anon="0") nfs:sys=(ro="*") 
            /export/install/patches        nfs=(anon="0") nfs:sys=(ro="*")
            /export/install/pkgs   nfs=(anon="0") nfs:sys=(ro="*") 
  zfs 
  root@testopen3:/export/install/pkgs/explo/5.13/sparc# share
  -               /opt/SUNWjet   anon=0,sec=sys,ro   "JET Framework" 
  -               /SHARED   anon=0   "" 
  -               /STAGE/Img/FLAR   anon=0,sec=sys,ro   "" 
  -               /dhcp   anon=0,sec=sys,ro   "" 
  -               /export/install/media/Solaris_10_1008X   anon=0,sec=sys,ro   "" 
  -               /export/install/media/Solaris_10_0508S   anon=0,sec=sys,ro ⁞  "" 
  -               /export/install/media/Solaris_10_1008S   anon=0,sec=sys,ro   "" 
  -               /export/install/EIS/Sep08   anon=0,sec=sys,ro   ""
  -               /rarp/Solaris_10_0508S/Solaris_10/Tools/Boot   anon=0,sec=sys,ro   "" 
  -               /export/install/patches   anon=0,sec=sys,ro   "" 
  -               /export/install/pkgs   anon=0,sec=sys,ro   "" 
  root@testopen3:/export/install/pkgs/explo/5.13/sparc# 
  root@testopen3:/export/install/pkgs/explo/5.13/sparc# cat /etc/dfs/dfstab 
  # Do not modify this file directly.
  # Use the sharemgr(1m) command for all share management 
  # This file is reconstructed and only maintained for backward 
  # compatibility. Configuration lines could be lost. 
  #
  #       share [-F fstype] [ -o options] [-d "<text>"] <pathname> [resource] 
  #       .e.g,
  #       share  -F nfs  -o rw=engineering  -d "home dirs"  /export/home2
  # 
  # 
  # 
  # Error: Syntax: share -F nfs -o ro,anon=0 /dhcp/OS_SOL_10_0508S/Solaris_10/Tools/Boot 
  # Error: Syntax: share -F nfs -o ro,anon=0 /dhcp/OS_SOL_10_1008S/Solaris_10/Tools/Boot 
  share -F nfs -o anon=0,sec=sys,ro -d "JET Framework" /opt/SUNWjet 
  share -F nfs -o anon=0 /SHARED
  share -F nfs -o anon=0,sec=sys,ro /STAGE/Img/FLAR 
  share -F nfs -o anon=0,sec=sys,ro /dhcp
  share -F nfs -o anon=0,sec=sys,ro /export/install/media/Solaris_10_1008X 
  share -F nfs -o anon=0,sec=sys,ro /export/install/media/Solaris_10_0508S 
  share -F nfs -o anon=0,sec=sys,ro /export/install/media/Solaris_10_1008S< 
  share -F nfs -o anon=0,sec=sys,ro /export/install/EIS/Sep08 
  share -F nfs -o anon=0,sec=sys,ro /rarp/Solaris_10_0508S/Solaris_10/Tools/Boot 
  share -F nfs /export/install/patches 
  share -F nfs /export/install/pkgs
  root@testopen3:/export/install/pkgs/explo/5.13/sparc#
 

11. Run make_client <client-name>

Here client-name=template-name. For the first time it will setup boot environment (DHCP, PXE).

This will populate:

• /etc/hosts
• /etc/bootparams
• DHCP server configuration
• Creates PXE boot environment

Boot type depends on "base_config_client_allocation="dhcp"" variable in client's template. If empty (="") the default is RARP (boot acording /etc/bootparams parameters supplied by /usr/sbin/rpc.bootparamd, which is Solaris default service svc:/network/rpc/bootparams:default.

E.G. client sf15k-c3

  root@testopen3:/opt/SUNWjet/Templates# grep alloca sf15k-c3
  base_config_client_allocation="dhcp"
  root@testopen3:/opt/SUNWjet/Templates#
  root@testopen3:/opt/SUNWjet/Templates# grep sf15k-c3 /etc/hosts
  10.18.139.35    sf15k-c3 
  root@testopen3:/opt/SUNWjet/Templates# grep sf15k-c3 /etc/bootparams 
  root@testopen3:/opt/SUNWjet/Templates# 
  root@testopen3:/opt/SUNWjet/Templates# grep sf15k-c3 /var/tmp/SUNWfiles1_10_18_139_0 
  10.18.139.35|010000BEA82270|03|10.18.139.80|4294967295|8345170109517529091|sf15k-c3| 
  root@testopen3:/opt/SUNWjet/Templates# 
  root@testopen3:/opt/SUNWjet/Templates# grep sf15k-c3 /var/tmp/SUNWfiles1_dhcptab 
  sf15k-c3|m|218706056904179713|:SrootPTH=/dhcp/OS_SOL_10_1008S/Solaris_10/Tools/Boot:SinstPTH=/dhcp/OS_SOL_10_1008S:SrootIP4=10.18.139.80:SrootNM=10.18.139.80:SinstIP4=10.18.139.80:SinstNM=10.18.139.80:SjumpsCF="10.18.139.80:/opt/SUNWjet":SsysidCF="10.18.139.80:/opt/SUNWjet/Clients/sf15k-c3":BootSrvA=10.18.139.80:BootFile=010000BEA82270:
  root@testopen3:/opt/SUNWjet/Templates# ls -l /dhcp/ 
  total 13 
  lrwxrwxrwx 1 root root    38 Nov  8 02:19 OS_SOL_10_0508S -> /export/install/media/Solaris_10_0508S 
  lrwxrwxrwx 1 root root    38 Nov  8 09:03 OS_SOL_10_1008S -> /export/install/media/Solaris_10_1008S 
  root@testopen3:/opt/SUNWjet/Templates#

You can see this is Sparc Solaris 10 10/2008 client booting from DHCP.

The default config files location is /var/tmp. You can check and manipulate DHCP server client entries and macros by pntadm(1) and dhtadm(1), not by hand.

E.G.:

Macro named "sf15k-c" deletion:

#dhtadm -D -m sf15k-c

Entry for client IP 10.18.139.35 in network 10.18.139.0 deletition:

 #pntadm -D 10.18.139.35 10.18.139.0

make_client script add them again if needed. You can need this if you want to install same client with same IP again, but with different hostname.

Fix any error otherwise you could waste your time on finding error during boot time.

12. Boot the client

Identify corret NIC to boot form, use can use watch-net-all in OBP to see at least where the ethernet link exists.

You cat see my sample client boot and install process HERE. This client had following TEMPLATE.

BE PATIENT! When booting sparc from DHCP it can takes minutes before you see any progress (first stage is TFTP download of inetboot, the HEX growing).

It usualy needs to go throught RARP process first and than restart DHCP process get IP.

E.G.:

Rebooting with command: boot net-boot:dhcp - install
Boot device: /pci@3c,700000/pci@1/network@0:dhcp  File and args: - install
Using Onboard transceiver - 100 Mbps FDX Link up
Timeout waiting for BOOTP/DHCP reply. Retrying ...
Timeout waiting for BOOTP/DHCP reply. Retrying ...
Timeout waiting for BOOTP/DHCP reply. Retrying ...
Timeout waiting for BOOTP/DHCP reply. Retrying ...
Timeout waiting for BOOTP/DHCP reply. Retrying ...
Timeout waiting for BOOTP/DHCP reply. Retrying ...
Timeout waiting for BOOTP/DHCP reply. Retrying ...
Timeout waiting for BOOTP/DHCP reply. Retrying ...
Timeout waiting for BOOTP/DHCP reply. Retrying ...
Timeout waiting for BOOTP/DHCP reply. Retrying ...
Failed to receive config params
Restarting DHCP process ...
4000 -

This can take 5minutes. If you are booting Solaris 10 update 6 it downloads boot-archive from boot server it can take another 15minutes - you will see only moving slash during that period.

You will probably see error "/dhcp/OS_SOL_10_1008S/Solaris_10/Tools/Boot failed" in your config, this can be ignored.

Timeout waiting for BOOTP/DHCP reply. Retrying ...
Timeout waiting for BOOTP/DHCP reply. Retrying ...
Failed to receive config params
Restarting DHCP process ...
4000 100 Mbps FDX Link up
boot: lookup /dhcp/OS_SOL_10_1008S/Solaris_10/Tools/Boot failed
SunOS Release 5.10 Version Generic_137137-09 64-bit
Copyright 1983-2008 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.

Now you can boot x86 via PXE, Sparc via BOOTP/RARP or DHCP. You can have DHCP server and bootparamd running both at the same time. PXE should default to GRUB install, where you can edit menu by your needs.

That's all hoping you had successful boot & install ;o)

This is note of fixing non-critical problem, DO NOT
USE unless want to play with net boot more & more
===================================================
THIS IS WR FOR INETBOOT LOOKUP FAILED ERROR
===================================================
* DHCP boot Sol10U6 ONLY!
~~~~~~~~~~~~~~~~~~~~~
        - SU for root
        - run /root/batch/pxestart.ksh
                +stops bootparamd
                +starts DHCP server
                +delete /dhcp/OS_SOL_10_1008S dir (my boot hack)
        - create subnet using "dhcpmgr" (run as root export DISPLAY first)
        - make_client XXXX (takes ages after first 3 lines for the first time)
        - "Passed" ?? => continue

        -----------------------------------------------------------------------------------------------
        OPTIONAL for correct inetboot: (otherwise not stopping error:
        "3a000 boot: lokup /dhcp/OS_SOL_...../Tools/Boot failed"
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

        - fix ":SinstPTH=" VAR in client's macro /w dhcpmgr
                +set to: "/export/install/media/Solaris_10_1008S"
                +leave notify DHCP server checkbox SET!

          -fix via CMD: dhtadm -M -m <client-name> -e SinstPTH=/export/install/media/Solaris_10_1008S
                        svcadm restart dhcp-server

        - grep  /var/tmp/SUNWfiles1_dhcptab
        - run /root/batch/fixSparcDHCPboot.ksh
        -----------------------------------------------------------------------------------------------


        - in OBP: boot net:dhcp - install
        - ignore "Timeout waiting for BOOTP/DHCP...." up to 10x of sf15k
        - wait for DHCP process restart + TFTP download
        - up to 30s for moving slash
        - up to 30min for boot-archive download
        - should be running then!

===================================================




root@testopen3:/opt/SUNWjet/Templates# cat /root/batch/pxestart.ksh
##mount /tftpboot/I86PC.Solaris_10-1
svcadm disable bootparams
svcadm enable dhcp-server
rm -r /dhcp/OS_SOL_10_1008S
root@testopen3:/opt/SUNWjet/Templates#
root@testopen3:/opt/SUNWjet/Templates# cat /root/batch/fixSparcDHCPboot.ksh

# This is JET created dir /w its links in it, but bad
cd /dhcp

# Remove bad link auto-generated by JET (make_client script)
rm OS_SOL_10_1008S

# Restore my structure which links to "SOL_10_1008S" dir here
tar xvf BackupOf-OS_SOL_10_1008S.tar
root@testopen3:/opt/SUNWjet/Templates#
root@testopen3:/opt/SUNWjet/Templates# tar tvf /dhcp/BackupOf-OS_SOL_10_1008S.tar
drwxr-xr-x root/root         0 2008-11-07 13:19 OS_SOL_10_1008S/
drwxr-xr-x root/root         0 2008-11-07 13:19 OS_SOL_10_1008S/Solaris_10/
drwxr-xr-x root/root         0 2008-11-07 13:19 OS_SOL_10_1008S/Solaris_10/Tools/
lrwxrwxrwx root/root         0 2008-11-07 13:55 OS_SOL_10_1008S/Solaris_10/Tools/Boot -> ../../../SOL_10_1008S/Boot/
root@testopen3:/opt/SUNWjet/Templates#
root@testopen3:/opt/SUNWjet/Templates# ls -lR /dhcp/SOL_10_1008S/
/dhcp/SOL_10_1008S/:
total 3
drwxr-xr-x 3 root staff 8 Oct 27 21:21 Boot
drwxr-xr-x 2 root root  4 Oct 27 21:21 boot

/dhcp/SOL_10_1008S/Boot:
total 81737
-rw-r--r-- 1 root root 72360652 Oct 27 20:49 X.cpio.bz2
-rw-r--r-- 1 root root   287275 Oct 27 20:49 X_small.cpio.bz2
-rw-r--r-- 1 root root  9780327 Oct 27 20:49 lu.cpio.bz2
-rw-r--r-- 1 root root     1530 Oct 27 20:49 lu.platforms
-rw-r--r-- 1 root root   829821 Oct 27 20:47 pkg_db.cpio.bz2
drwxr-xr-x 5 root root        5 Oct 27 21:21 platform

/dhcp/SOL_10_1008S/Boot/platform:
total 5
drwxr-xr-x 2 root root 5 Nov  7 13:09 sun4u
drwxr-xr-x 2 root root 5 Nov  7 13:57 sun4us
drwxr-xr-x 2 root root 5 Nov  7 13:58 sun4v

/dhcp/SOL_10_1008S/Boot/platform/sun4u:
total 1414
lrwxrwxrwx 1 root root      28 Nov  7 13:09 boot_archive -> ../../../boot/sparc.miniroot
-rw-r--r-- 1 root root  221280 Oct 27 20:49 inetboot
-rw-r--r-- 1 root root 1084048 Oct 27 20:49 wanboot

/dhcp/SOL_10_1008S/Boot/platform/sun4us:
total 1414
lrwxrwxrwx 1 root root      28 Nov  7 13:57 boot_archive -> ../../../boot/sparc.miniroot
-rw-r--r-- 1 root root  221304 Oct 27 20:49 inetboot
-rw-r--r-- 1 root root 1084064 Oct 27 20:49 wanboot

/dhcp/SOL_10_1008S/Boot/platform/sun4v:
total 1414
lrwxrwxrwx 1 root root      28 Nov  7 13:58 boot_archive -> ../../../boot/sparc.miniroot
-rw-r--r-- 1 root root  220536 Oct 27 20:49 inetboot
-rw-r--r-- 1 root root 1083280 Oct 27 20:49 wanboot

/dhcp/SOL_10_1008S/boot:
total 182547
-rw-r--r-- 1 root root      8192 Oct 27 20:49 hsfs.bootblock
-rw-r--r-T 1 root root 186777600 Oct 27 20:54 sparc.miniroot
root@testopen3:/opt/SUNWjet/Templates#
 

NET BOOT SAMPLE IS HERE.

DISK BOOT SAMPLE IS HERE.

Posted at 04:50odp. XI 08, 2008 by Arrowaver in Sun  |  Comments[4]

Live upgrade (ABE) with local zones and SVM mirroring and aplication of patches

You can find full commented example of use Solaris Volume Manager & whole-root Solaris zones with Live Upgrade technology for patching HERE.

Enjoy

Arrowaver

Posted at 12:22odp. XI 08, 2008 by Arrowaver in Sun  |  Comments[0]