The Blog of The MattMan ! Mattman's Blog

This is a Quick 'n' Dirty guide to creating an LiveCD ISO image for OpenSolaris on OpenSolaris.

NOTE: these notes are written assuming you have access to Sun's internal servers. You may be able to populate your own local IPS repo from some other source, I don't know how off hand but if you find out let me know and I'll add that info here aswell.

1. Some Environment setup

Install ss-dev and SUNWonbld packages, this will ensure all required dev tools are installed such as SunStudioExpress and create SUNWspro symbolic link :

  $ pfexec pkg install ss-dev SUNWonbld
  $ pfexec ln -s /opt/SunStudioExpress /opt/SUNWspro

Some other dependencies that I also installed were :

  $ pfexec pkg install SUNWgnome-common-devel
  $ pfexec pkg install SUNWpython-setuptools
  $ pfexec pkg install SUNWpython-pycurl

If the build fails, see what the build was looking for and use pkg search filename to see what other package might be necessary.

Ensure your PATH has SunStudioExpress and onbld included, and also has /usr/bin before /usr/gnu/bin :

  $ export PATH=/usr/bin:/opt/SunStudioExpress/bin:/opt/onbld/bin:$PATH

2. Create local IPS Repository

First clone the gate (IPS) source and build/install it :

  $ cd
  $ hg clone ssh://anon@hg.opensolaris.org/hg/pkg/gate
  $ cd ~/gate/src
  $ pfexec dmake packages

All gate related packages are available under : ~/gate/packages/i386.

On OpenSolaris, just enure you have the most up to date versions of these package installed from your default authority :

  $ pfexec pkg install SUNWipkg SUNWipkg-brand SUNWipkg-gui SUNWipkg-gui-data  SUNWipkg-gui-l10n SUNWipkg-um SUNWpython-cherrypy SUNWpython-mako SUNWpython-ply SUNWpython-pycurl

On SXDE you should remove the old versions of these packages and install the newly built ones. Before installing these packages ensure the pkg/server SMF service is disabled.

  $ svcs -a | grep pkg/server
  online         Feb_18   svc:/application/pkg/server:default

If online then disable :

  $ pfexec svcadm disable pkg/server
  $ svcs -a | grep pkg/server:default
  disabled       11:26:58 svc:/application/pkg/server:default

Now as root :

    $ pkgrm SUNWipkg
    $ cd ~gate/packages/i386
    $ ls
    SUNWipkg               SUNWipkg-gui           SUNWipkg-gui-l10n        
    SUNWpython-cherrypy    SUNWpython-ply         pkg5-c700981b0af2.pkg
    SUNWipkg-brand         SUNWipkg-gui-data      SUNWipkg-um              
    SUNWpython-mako        SUNWpython-pycurl      pkg5.pkg
    $ pkgadd -d pkg5.pkg
    $ pkginfo SUNWipkg

3. Set up Local IPS Repository.

By default pkg/server uses /var/pkg/repo for it's package directory. If you choose to use the default the first time you import just ensure it's contents are clear out :

  $ svcadm disable pkg/server
  $ pfexec rm -rf /var/pkg/repo/*

Best practice though would be to create a new zfs location for local repo:

  $ pfexec zfs create rpool/ips
  $ pfexec zfs create rpool/ips/repo
  $ pfexec zfs set mountpoint=/export/ips/repo rpool/ips/repo

If you do create a new directory you need to set the pkg/inst_root property of the SMF service to point to this new location :

  $ pfexec svccfg -s pkg/server setprop pkg/inst_root = /export/ips/repo

You need to update the default port number from 80 to 10000, 10000 is the port used by distro-import :

  $ pfexec svccfg -s pkg/server setprop pkg/port = 10000

Refresh and enable pkg/server, and check your settings :

  $ pfexec scvadm refresh pkg/server
  $ pfexec svcadm enable pkg/server
  $ svcprop pkg/server | grep port
  pkg/port count 10000
  $ svcprop pkg/server | grep inst_root
  /export/ips/repo (or /var/pkg/repo)

You can browse the repo in browser with following URL :

     http://localhost:10000

4. Create your own slim install packages.

Get the sources:

  $ cd
  $ hg clone ssh://anon@hg.opensolaris.org/hg/caiman/slim_source

Read ~/slim_source/usr/src/README and follow the instructions on how to build.

You need three other packages not on the IPS servers but are available on http://dlc.sun.com/downloads. These are SUNWzoneint, SUNWwbint, and SUNWldskint. Just download and install them using pkgadd. The following shows a single line command that will download and install in one go :

  $ pfexec pkgadd -d http://dlc.sun.com/osol/install/downloads/current/SUNWldskint.`uname -p`.pkg all
  $ pfexec pkgadd -d http://dlc.sun.com/osol/install/downloads/current/SUNWwbint.`uname -p`.pkg all
  $ pfexec pkgadd -d http://dlc.sun.com/osol/install/downloads/current/SUNWzoneint.`uname -p`.pkg all

Before you build if you have pulled your slim source to a location other than the default "slim_source", you need modify developer.sh and change CODEMGR_WS to point to this location. To build your packages just :

  $ cd ~/slim_source/usr/src
  $ ./nightly developer.sh

This will build packages under : ~/slim_source/packages/i386/nightly-nd, and the nightly build log is located in ../../log/*/nightly.log.

5. Import packages to local Repo.

Now that we have some custom packages we want to generate a local repo which will contain latest nevada and your custom packages. To import latest nevada packages to your local IPS repo you need access to Sun internal servers. You can edit the Makefile located at :

    ~/gate/src/util/distro-import/Makefile

And set the following variables :

NOTE : If you don't have any custom packages leave TEST_PKGS blank.

Save and exit the Makefile. The following manifest file can be used to initialize your pkg/server service back to default values. Before importing you should edit the file and ensure the pkg/inst_root and pkg/port are assigned the correct values e.g. /export/ips/repo and 10000 :

  $ cd ~/gate/src/svc
  $ pfexec svcadm disable pkg/server
  $ pfexec svccfg import pkg-server.xml
  $ pfexec svcadm refresh pkg/server
  $ pfexec svcadm enable pkg/server

Change directory back into distro-import, and using a the specific build number use redist_import script to populate your local IPS repo. e.g. for build 121 :

  $ cd ~/gate/src/util/distro-import
  $ make 121/redist_import

NOTE:This will fail if you have the JDS CBE included on your PATH, e.g. ensure /opt/dtbld/bin is NOT included in your PATH.

This step can take quite a while. In your browser, keep an eye on http://localhost:10000 — it starts get new packages after a while.

After an initial import to your local IPS repo, if you created a new zfs filesystem for your local repo, it's best to create a snapshot at this time, this will allow you to return to a clean initial repo without haveing to re-import everything again :

  $ pfexec zfs snapshot rpool/ips/repo@virgin 

To roolback to this snapshot at a later time you can run the following :

  $ pfexec zfs rollback rpool/ips/repo@virgin 

There are some alternative imports which can be run but they have caveats:

6. Install or Push local test packages into your local repo.

There are a number of methods that can be used to push some custom packages into your local repo.

• 1. Use solaris.py to push into your local repo
  

  To push custom SUNWgui-install into http://localhost:10000 repo :
  

    $ cd ~/gate/src/util/distro-import/
    $ mkdir gui-install
    $ cat > gui-install/SUNWgui-install << EOF
    > package SUNWgui-install
    > import SUNWgui-install
    > end package
    > EOF
  

  The above script may also contain other commands if required such as adding users/groups which might be required by specific packages. See ~/gate/src/utils/distro-import/105/common/SUNWslim-utils as an example of how to create a user. Now create a metafile pointing to the gui-install stuff :
  

    $ cd ~/gate/src/util/distro-import/
    $ cat > gui-install.list << EOF
    > include gui-install/SUNWgui-install
    > EOF
  

  Finally run solaris.py to push the SUNWslim-utils package into the repo :
  

    $ ./solaris.py -b 6.6 -d -s http://localhost:10000 -w ~/slim_source/packages/i386/nightly-nd slim-utils.list
  

  Pay attention to the above. The "-b 6.6" specifies the revision. You just bump it up higher and higher to make sure that version is found. The ~/slim_source/packages/i386/nightly-nd is the directory containing the packages.
  

  You can verify that the right version was pushed with the following :
  

    $ pkg image-create -F -a test=http://localhost:10000 /tmp/test-image
  
    $ pkg -R /tmp/test-image install SUNWgui-install
    DOWNLOAD                                  PKGS       FILES    XFER (MB)
    Completed                                  1/1       30/30      0.0/0.0
  
    PHASE                                        ACTIONS
    Install Phase                                  58/58
    PHASE                                          ITEMS
    Reading Existing Index                           8/8
    Indexing Packages                                1/1
  
    $ pkg -R /tmp/test-image list -av SUNWgui-install
    FMRI                                                           STATE      UFIX
    pkg:/SUNWgui-install@0.5.11,5.11-6.6:20090219T082311Z           installed  ----
  

  Note the -6.6 at the end of the version shown in the list above.
  

  In order for distro_const to include the correct version of your custom package in the ISO, the package entire needs to be rebuilt so that it's manifest includes your package version.
  

  To achieve this firstly go back to your IPS gate source and edit Makefile :
  

    $ cd ~/gate/src/util/distro-import
    $ vi Makefile
  

  Edit the Makefile, and comment out the recreation of entire.incorporation, by changing line :
  

        From : entire: $(BUILDID)/entire.incorporation
        To   : entire: #$(BUILDID)/entire.incorporation
  

  Now edit the specific build entire.incorporation file :
  

    $ vi 121/entire.incorporation
  

  and change the version of the package listed here to your local version e.g. for SUNWgui-install :
  

        From : depend fmri=SUNWgui-install@0.5.11-0.121 type=incorporate
        To   : depend fmri=SUNWgui-install@0.5.11-6.6 type=incorporate
  

  Now re-build the entire package :
  

    $ make 121/entire
    PKG_REPO=http://localhost:10000 ./import_manifest_file \
        entire@0.5.11,5.11-0.`echo 121 | tr -d '[a-z]'` \
        121/entire.incorporation
    PUBLISHED
    pkg:/entire@0.5.11,5.11-0.121:20090903T143856Z
  

  Do a quick restart of the pkg/server, even though I'm not sure it's definitely required. Now check http://localhost:10000, search for package entire, once found click the "manifest" link and browse down to your custom package, the version listed here should be the one you published.

• 3. Import at initial local repo creation time
  

  When your first populate your local IPS repo you could set TEST_PKGS as stated in the section above to point to the location of your custom SVR4 packages, it will automatically grab all packages here and install them in your local repo.

• 4. Re-run redist_import to re-populate your local repo.
  

  Quite similar to previous method, except in this case you already have a populated local repo, you can re-run redist_repo to re-populate your local IPS repo. Before doing so you can set the environment variable TEST_PKGS to point to your custom packages :
  

    $ export TEST_PKGS=~/slim_source/packages/i386/nightly-nd
  

  Then re-run redist_import specifying -e make flag to override environment variables in the Makefile from current shell environment:
  

    $ cd ~/gate/src/util/distro_import
    $ /usr/bin/make -e 122/redist_import
  

NOTE:Options 2 & 3 above can take quite a bit of time depending on network connectivity and hardware specification.

7. Build Distribution ISO.

Ensure SUNWdistro-const is installed :

  $ pfexec pkg install SUNWdistro-const

Now make a copy of the distro XML manifest that you want to use to generate the ISO :

  $ mkdir ~/mydistro
  $ cp /usr/share/distro_const/slim_cd/all_lang_slim_cd_x86.xml ~/mydistro/mydistro.xml
  $ cd ~/mydistro

Now edit ~/mydistro/mydistro.xml :

• Change the distribution name to one of your choice :
  

       From : <distribution name="OpenSolaris">
       To   : <distribution name="MyDistro">
  

• Change url element of <pkg_repo_default_authority> section from :
  

        From : pkg.opensolaris.org/release
        To   : localhost:10000
  

• Change authname element of <pkg_repo_default_authority> section from :
  

        From : opensolaris.org
        To   : myrepo
  

• Optional to remove generation of usb image.
  If you don't want the distro.usb image to be generated then comment out or remove the usb finalizer script from within mydistro.xml. Go to section <finalizer> section and remove/comment out the following <finalizer> section :

     <script name="/usr/share/distro_const/create_usb">
        <checkpoint
           name="usb"
           message="USB image creation"/>
     </script>
  

• If you are using a custom finalizer script to pkgadd custom packages as detailed in the last section below, ensure the required changes are made to the <finalizer> section.

Now run distro_const on this mydistro.xml file to generate your ISO :

  $ pfexec distro_const build ~/mydistro/mydistro.xml

By default this will generate an installation under ZFS dataset :

    rpool/dc

If this filesystem does not exist it will be created. The media once generated will exist in :

    /rpool/dc/media

8. Test Your LiveCD ISO Using VirtualBox.

Boot from /rpool/dc/media/MyDistro.iso.
9. Use of Custom Finalizer script to install custom packages.

An alternative to pushing custom packages into your local repo, and regenerateing the entire package, you could simply use a finalizer script to add these packages to distro_const's default pkg_image area when it is creating the image.

When distro_const is run to create your distro, a number of finalizer scripts are run for the various steps to generate the distribution ISO image. These scripts are defined in the <'finalizer> section within the xml manifest used by distro_const. e.g. mydistro.xml mentioned above.

You can add your own finalizer script to pkgadd your custom packages to the image install area before the distro itself is constructed. The first script is "pre_bootroot_pkg_image_mod", add your custom finalizer install script just after this and just before the "slimcd_pre_bootroot_pkg_image_mod" script. It might look like the following :

   <script name="~/mydistro/add-custom-pkgs">
      <checkpoint 
         name="add-custom-pkgs" 
         message="Add Custom pkgs"/>
   </script>

The script add-custom-pkgs might contain the following :

  $ cat ~/mydistro/add-custom-pkgs
  #!/usr/bin/ksh93

  ADMIN_FILE=/tmp/admin.$$
  cat << \ADMIN_EOF > $ADMIN_FILE
  mail=
  instance=unique
  partial=nocheck
  runlevel=nocheck
  idepend=nocheck
  rdepend=nocheck
  space=nocheck
  setuid=nocheck
  conflict=nocheck
  action=nocheck
  networktimeout=60
  networkretries=3
  authentication=quit
  keystore=/var/sadm/security
  proxy=
  basedir=default
  ADMIN_EOF

  pkgadd  -a ${ADMIN_FILE}  -d ~/slim_source/packages/i386/nightly-nd -R /rpool/dc/build_data/pkg_image SUNWgui-install

This script simply takes a custom SVR4 package version of SUNWgui-install which I have built in ~/slim_source/packages/i386/nightly-nd and installs it to the default package image area /rpool/dc/build_data/pkg_image.

Once you've edited your manifest xml file and added your custom finalizer script entry, just re-run distro_const to generate your ISO as specified in the previous section.

10. Summary And Links

Well done you made it to the end of what turned out to be a lot longer that I had originally planned. If you find errors in the above, please let me know and I will update the blog.

Some resources that I used when putting this blog together :

• Do It Yourself Distro Constructor Notes (SXDE)