Connecting the dots with Solaris and middleware In the middle

This entry addresses how to use the OpenSolaris JeOS prototype and VirtualBox as the basis for deploying an Automated Installer (AI) environment.  A follow-on entry addresses the use of a custom AI manifest to install a greatly reduced form of OpenSolaris using this environment.

As part of the JeOS project's effort to spur an initial specification and experimental implementation of OpenSolaris base install profiles, I wanted to be able to easily generate from scratch installations in a VirtualBox environment hosted on my Mac OS X system.  Since I needed only a headless, text oriented installation of OpenSolaris to run the Automated Installer's (AI's) install service, I used the 2009.06 JeOS prototype VM image as the platform for the AI install server.

While looking for a VirtualBox-based AI example, I found Chris Armes' blog, Configuring an auto install client / server setup using VirtualBox. (I've since ran into another example here and a bare metal example here). The primary difference between Chris' environment and the example below is that I started with the reduced JeOS installation of OpenSolaris as compared to a Live CD-based installation. I also ended up using development build 126 of OpenSolaris as compared to the 2009.06 release.

Starting with a greatly reduced installation of OpenSolaris for the AI server meant that I had to spend a fair amount of time troubleshooting missing dependencies that weren't an issue when starting with a Live CD-based installation. (I'll have to review my notes and do some more digging to see if any of those dependencies warrant filing of new bugs).

In addition to referring to Chris' and the other examples, I referred to the official Automated Installer documentation. 

Here are the overall steps that I took to  get this configuration up and running:

1. Downloaded and Deployed JeOS Prototype to VirtualBox
   
2. Installed Build 126 AI and AI ISO Image
3. Configured the AI Server
   
4. Created an Install Service
   
5. Created an AI Client in VirtualBox
6. Booted the AI Client and Installed OpenSolaris
   

Details of these steps follow.

1. Downloaded and Deployed JeOS Prototype to VirtualBox

• Deployed 2009.06 JeOS prototype VM image for VirtualBox.
• Customized the installation based on the Next Steps in the JeOS documentation:
  
• Streamlined the boot experience
• Upgraded to the latest development build, 2010.03 build 126
• Added a personal user ID with administrator privileges
• Set up password-based SSH access
  
• Defined static IP address for e1000g0 interface
  

2. Installed Build 126 AI and AI ISO Image

$ pfexec pkg install SUNWinstalladm-tools

Download the build 126 AI ISO image from Genunix:

/export/aiimages/osol-1002-126-ai-x86.iso

3. Configured the AI Server

e1000g0 is my install server's external network interface while e1000g1 is a newly added internal network interface. The install clients will talk to the AI server via this internal interface.

In VirtualBox, defined a second network adapter for the install server VM. Set it to use the "Internal network" and restarted the VM.

Set up the hostname file for internal interface:

$ cat /etc/hostname.e1000g1
192.168.3.50

Added an entry for the internal network to the netmasks file:

$ cat /etc/netmasks
192.168.2.0 255.255.255.0
192.168.3.0 255.255.255.0

Configured and brought up the internal interface:

$ pfexec ifconfig e1000g1 inet 192.168.3.50 netmask 255.255.255.0 broadcast 192.168.3.0

$ pfexec ifconfig e1000g1 up

Checked the resulting interfaces:

$ ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
e1000g0: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu 1500 index 2
        inet 192.168.2.10 netmask ffffff00 broadcast 192.168.2.255
e1000g1: flags=1100843<UP,BROADCAST,RUNNING,MULTICAST,ROUTER,IPv4> mtu 1500 index 3
        inet 192.168.3.50 netmask ffffff00 broadcast 192.168.3.255
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
        inet6 ::1/128

Enabled port forwarding:

$ pfexec routeadm -e ipv4-forwarding -u

Note that routing services are not enabled since we're using static routing: 

$ pfexec routeadm
              Configuration   Current              Current
                     Option   Configuration        System State
---------------------------------------------------------------
               IPv4 routing   disabled             disabled
               IPv6 routing   disabled             disabled
            IPv4 forwarding   enabled              enabled
            IPv6 forwarding   disabled             disabled

           Routing services   "route:default ripng:default"

Routing daemons:

                      STATE   FMRI
                   disabled   svc:/network/routing/route:default
                   disabled   svc:/network/routing/rdisc:default
                   disabled   svc:/network/routing/legacy-routing:ipv4
                   disabled   svc:/network/routing/legacy-routing:ipv6
                     online   svc:/network/routing/ndp:default
                   disabled   svc:/network/routing/ripng:default

Enabled NAT to allow AI clients on the private network to access Internet-based package repositories (e.g. http://pkg.opensolaris.org/dev)  and for any troubleshooting activity to access other systems on the 192.168.2.0 network.

Defined IP filter rules (wide open for now):

$ cat ipf.conf

pass  in     quick on e1000g0 all
pass  out    quick on e1000g0 all
pass  in     quick on e1000g1 all
pass  out    quick on e1000g1 all

Applied IP filter to network interfaces:

$ cat pfil.ap
e1000g     -1      0       pfil

Defined NAT mapping:

$ cat ipnat.conf
map e1000g0 192.168.3.0/24 -> 192.168.2.10/32 portmap tcp/udp 10000:65000
map e1000g0 192.168.3.0/24 -> 192.168.2.10/32

Ensured IP filter service was online:

$ pfexec svcadm enable ipfilter

$ svcs -a | grep ipfilter
online         14:26:27 svc:/network/ipfilter:default

Ensured IP filter rules were loaded:

$ pfexec ipf -Fa -f /etc/ipf/ipf.conf

$ pfexec ipfstat -ionh
32 @1 pass out quick on e1000g0 all
0 @2 pass out quick on e1000g1 all
62 @1 pass in quick on e1000g0 all
0 @2 pass in quick on e1000g1 all

Enabled multicast service:

$ pfexec svcadm enable dns/multicast
$ pfexec svcs -a | grep dns/multicast
online         15:50:08 svc:/network/dns/multicast:default

Enabled inetd service:

$ pfexec svcadm enable /network/inetd:default
$ pfexec svcs -a | grep inetd:default
online         17:02:19 svc:/network/inetd:default

Since the AI server requires a DHCP service and the DHCP service needs Java, installed the JRE:

$ pfexec pkg install SUNWj6rt

4. Created an Install Service

Listed the install clients in the hosts file:

$ cat /etc/hosts

::1 localhost
127.0.0.1 localhost
192.168.2.10 opensolaris opensolaris.local loghost
192.168.3.50 aiserver
192.168.3.60 aiclient0
192.168.3.61 aiclient1
192.168.3.62 aiclient2
192.168.3.63 aiclient3

Created the install service for these clients:

$ pfexec installadm create-service -n 1002-126-x86 -i 192.168.3.60 -c 4 -s /export/aiimages/osol-1002-126-ai-x86.iso /export/aiserver/osol-1002-126-ai-x86

$ installadm list
The install services running on the system are:

1002-126-x86
1002-126-x86

Ensured DHCP server was set up properly:

$ pfexec /usr/sbin/dhtadm -M -m 192.168.3.0 -e  Router=192.168.3.50 -g

$ pfexec  dhtadm -P
Name                    Type            Value
==================================================
dhcp_macro_1002-126-x86 Macro           :Include=osol:BootSrvA=192.168.2.10:BootFile=1002-126-x86:GrubMenu=menu.lst.1002-126-x86:
192.168.3.0             Macro           :Subnet=255.255.255.0:RDiscvyF=1:Broadcst=192.168.3.255:Router=192.168.3.50:
osol                    Macro           :Include=Locale:Timeserv=192.168.2.10:LeaseTim=86400:LeaseNeg:DNSdmain="kampmeier.com":DNSserv=66.112.11.87 66.112.11.88:
Locale                  Macro           :UTCoffst=0:
GrubMenu                Symbol          Site,150,ASCII,1,0

Next Steps

At this stage I pretty much followed the instructions in steps 9, 10 and 11 of Chris' blog entry.

The main difference is that I customized the default AI manifest to point at the development repository, http://pkg.opensolaris.org/dev,  rather than the default release repository. See the How to Replace the Default AI Manifest documentation for details on this process.

Caution: When I ran the following command to replace the default manifest:

$ pfexec /usr/sbin/installadm add -m new-default.xml -n 1002-126-x86

I had expected the following file to have been updated:

/export/aiserver/osol-1002-126-ai-x86/auto_install/default.xml

But that's not the case.  It seems as though that file is a reference default file and the real default file in use by the install service is housed under:

/var/ai/<install service port number>/AI_data/default.xml

The lack of any overt feedback from the installadm command and not seeing any change to the other default.xml file led me to believe that something had failed.

By default the AI installation will yield a desktop installation of OpenSolaris.  In the next blog entry I've described how to use a custom manifest to install a greatly reduced, text-based headless server installation of OpenSolaris using this JeOS and VirtualBox-based AI environment.