Live from New Jersey...

Update to 2006 Blueprint For those who are looking for the findbdev program mentioned on pages 19 and 20 of my Blueprint article GRUB and the Solaris Operating System on x86 Platforms - A Guide to Creating a Customized Boot DVD, the Sun download site has changed internally to the point that the URL for the zip file that contains findbdev, mntusr.xml, and mntusr is no longer valid. The new link is here. (2009-11-08 07:06:31.0) Permalink

Creating a customized Solaris boot DVD

Starting with Solaris 10u6 and Solaris Nevada build 80, the mechanics of booting have changed drastically for SPARC. Without going into the details, suffice it to say that the boot process has become very similar to that in Solaris x86. Because of this, the process of creating a customized boot DVD is also radically different. The good news is that the process is much simpler now, and there are utilities in Solaris to do large chunks of the work. In addition, the procedure is essentially the same for SPARC and x86, the only difference being the mkisofs command line used to recreate the ISO image at the end.

I will attempt to give a detailed description here on how to extract and rebuild the DVD image. I'll also go through the steps needed to set the customized image up to do a hands-off jumpstart installation. It will be necessary to perform this procedure on a system running Solaris 10u6 or newer as there are a number of utilities needed that didn't exist prior to this release.

Extracting the image

Previous versions (< s10u6) of the SPARC Solaris installation DVD had a number of VTOC slices on it. The first slice was a High Sierra File System (HSFS) and contained all the installation packages along with the jumpstart tools. The second slice contained a bootable UFS miniroot. Slice 2,3,4,5, and 6 contained the boot blocks for the sun4c, sun4m, sun4d, sun4u, and sun4v architectures respectively. This scheme is no longer used as of Solaris 10u6. There are still eight slices on the DVD, but this is merely maintained for backward compatibility with OpenBoot PROM and all eight slices point to the entire contents of the DVD.

Because of this, extraction of the contents of the DVD is much simpler. A cpio of the mounted DVD will suffice. Assuming that vold is running, it has mounted the DVD as /cdrom/cdrom0, and we want to copy it to /var/tmp/dvd, you can issue the following commands to copy the image:

# mkdir /var/tmp/dvd
# cd /cdrom/cdrom0
# find . -depth -print | cpio -pdm /var/tmp/dvd

When the copy is done, you will have effectively staged the DVD image for customization.

One note about vold. I mentioned before that the new form of the SPARC Solaris DVD contains eight slices. However, vold will not show you this. The vold has code in it to compare all the slices on a removable medium and ignore any duplicates, opting to only mount the first unique slice it finds. In fact, there is a bug in this code that causes vold to core dump when trying to calculate which slices are duplicates. The effect is that when you place a SPARC Solaris 10u6 DVD in the drive on a SPARC system with vold running, vold will go into a core dump loop as SMF continually tries to restart the vold service. This bug is fixed in patch 139973-01.

If using an ISO image file instead of a physical DVD, the entire contents of the DVD can be accessed by using lofiadm and mounting the lofi device:

# lofiadm -a `pwd`/sol-10-u6-ga1-sparc-dvd.iso
/dev/lofi/1
# mount -F hsfs -o ro /dev/lofi/1 /mnt

At this point, you can extract the DVD contents to a directory

# mkdir /var/tmp/dvd
# cd /mnt
# find . -depth -print | cpio -pdm /var/tmp/dvd

Unmount and remove your lofi device as it's no longer needed

# cd /
# umount /mnt
lofiadm -d /dev/lofi/1

Extracting the miniroot

As mentioned, the SPARC miniroot was previously maintained as a separate UFS slice on the DVD. It is now maintained as a single file in the /boot directory, just like x86. By convention, the file name for the sparc miniroot is (cleverly enough) sparc.miniroot. Extracting this miniroot is a simple matter. There is a new utility in SPARC Solaris as of S10u6 to do this called root_archive. The root_archive script has four options to it:

• pack - take a root directory and pack it into a miniroot image file
• unpack - take a miniroot image file and extract it to a directory
• packmedia - same as pack, except that the package database and other data not necessary for booting is stripped out of the miniroot and bundled together in a compressed tar file to the Solaris_10/Tools/Boot directory
• unpackmedia - same as unpack except that the package database and other data is also extracted from the Solaris_10/Tools/Boot directory on the DVD image

We will be using the packmedia and unpackmedia options here. 

To use root_archive to extract your miniroot, perform the steps below. This assumes that you staged your DVD copy in /var/tmp/dvd and you are staging your miniroot copy to /var/tmp/mrdir.

# /boot/solaris/bin/root_archive unpackmedia /var/tmp/dvd /var/tmp/mrdir

Before we go further, I need to mention bug 6738836 which causes the umount and lofiadm -d in the root_archive script to fail to clean up after itself correctly. When I ran this command for this example, this is what I got:

umount: /tmp/mnt3553 busy
rmdir: directory "/tmp/mnt3553": Directory is a mount point or in use
lofiadm: could not unmap file /var/tmp/dvd/boot/sparc.miniroot: Device busy
rmdir: directory "/tmp/mnt3553": Directory is a mount point or in use

The workaround to this is to forceably unmount the mount point and delete the lofi dev manually:

# lofiadm
Block Device             File
/dev/lofi/1              /var/tmp/dvd/boot/sparc.miniroot
# umount -f /tmp/mnt3553
# lofiadm -d /dev/lofi/1

This bug only affects SPARC Solaris.

Modifying the miniroot

At this point, modifications to the miniroot can be made to /var/tmp/mrdir. Since we used the unpackmedia option to root_archive, the package database is intact and packages and patches can be installed. Also, for jumpstart, you can place your sysidcfg file in /etc in the miniroot, after first removing the symbolic link that exists there.

Modifying the DVD image for jumpstart

If you're creating this DVD image to do a hands off installation using jumpstart, there are a number of modifications you'll need to make to the DVD image (/var/tmp/dvd). First, you'll need to tell jumpstart that the necessary files are available on the DVD itself. Do this by executing the following command:

# touch /var/tmp/mrdir/.tmp_proto/.preinstall

Next, you'll need to put your profile, rules files, begin and finish scripts in the directory /var/tmp/dvd/Solaris_10/Misc/.install_config. Once this is done, run check on your rules file to create the rules.ok file:

# cd /var/tmp/dvd/.install_config
#  /var/tmp/dvd/Solaris_10/Misc/jumpstart_sample/check
Validating rules...
Validating profile profile...
The custom JumpStart configuration is ok.

If performing a flash archive installation, you can put your flash archive anywhere on the DVD image - /var/tmp/dvd. When the system is booted off your custom DVD, the top-level directory will be /cdrom. The lines in your jumpstart profile that reference the flash archive would look something like this:

install_type        flash_install
archive_location    local_file /cdrom/flasharchive

This assumes you put the flash archive in the top level directory of your DVD image.

Since flash archives take up a large amount of space, you'll most likely need to free some space up on the DVD to make room for it. Normally, a flash archive installation doesn't install packages from the Solaris medium so you can safely remove the Solaris_10/Product directory. Doing so should free up a sufficient amount of space to include the flash archive.

If you're performing a standard installation, then nothing else needs to be done other than to specify the metacluster you're installing in your profile.

Putting it back together

Once you've made all the necessary modifications to your miniroot, you'll need to reinstall the miniroot back on to the DVD image. Again, we'll use the root_archive program:

# /boot/solaris/bin/root_archive packmedia /var/tmp/dvd /var/tmp/mrdir

Once you run packmedia against your miniroot directory (/var/tmp/mrdir),  do not reuse it. This is because packmedia makes modifications to it that can only be undone by unpackmedia against the miniroot image file, which is now in /var/tmp/dvd. The prudent thing to do at this point is to remove it:

# rm -r /var/tmp/mrdir

At this point, we have the directory structure needed to create our ISO image file. Use mkisofs to create the image for burning. For SPARC, the following can be used:

# /usr/bin/mkisofs -N -D -R -d -l -J    \
    -G /var/tmp/dvd/boot/hsfs.bootblock \ 
    -B …                                \
    -graft-points                       \
    -relaxed-filenames                  \
    -V "My_DVD_Name"                    \ 
    -o /var/tmp/newdvd.iso              \ 
    /var/tmp/dvd

As mentioned before, the Solaris DVD will contain 8 identical slices. It's the -B ... (dash B space dot dot dot) option that creates the VTOC structure this way.

 For x86, the following command can be used to create the ISO image:

# /usr/bin/mkisofs -N -D -R -U -l -J  \
    -b boot/grub/stage2_eltorito    \
    -no-emul-boot                   \
    -boot-load-size 4               \
    -boot-info-table                \
    -c .catalog                     \
    -V "My_DVD_Name"                \
    -o /var/tmp/newdvd.iso          \
    /var/tmp/dvd

You now have an ISO image that can be burned to DVD. In Solaris, this would be accomplished using cdrw:

# cdrw -i -d /dev/rdsk/cxtxdxs2 /var/tmp/newdvd.iso

Assuming you have the sysidcfg, profile, and rules.ok files populated correctly, booting this DVD should perform a hand-off installation on the system that it's booted on.

(2009-01-23 15:22:39.0) Permalink Comments [7]

Booting SPARC and x86 Solaris from a single DVD

One of the projects I've been working on at Sun for the past 6 or 7 years is a utility CD that our field service engineers can boot on our customer's systems for the purpose of troubleshooting in a non-intrusive manner. Shortly after Solaris 10 came out, I started toying with the idea of creating a dual boot CD. This would be a Solaris CD that could be booted on both SPARC and x86 systems. I explored this possibility for a while until I came to a technical roadblock. There was just no way to make the x86 MBR co-exist with the SPARC VTOC on sector 0 of the CD, so I dropped the idea.

The other day, I started thinking about this idea again and realized that the last time I had tried this was when Solaris x86 was still using the Device Configuration Assistant as the boot loader. This mechanism required there to be an fdisk partition table in place. However, today's Solaris x86 has no such requirement as it uses GRUB as it's boot loader. GRUB (stage2_eltorito to be exact) is perfectly happy to use the ISO9660 structure of the CD to find and load the miniroot. Since ISO9660 doesn't do anything with the first logical cylinder (first 327680 bytes) of the CD, and El Torito doesn't use anything in the first 32K of the CD, it's possible to put arbitrary data in it without breaking anything on the x86 CD/DVD image.

The arbitrary data in this case can be a SPARC VTOC on sector 0. Because of this, it's possible to create a CD or DVD that can boot Solaris on both SPARC and x86. The general idea is:

• Make a copy of the x86 ISO image
• Pad this copy with 0's up to the next cylinder (327680 byte) boundary.
• Append the SPARC ISO image to it.
• Take the VTOC from the SPARC image and adjust the starting cylinder (dkl_cylno - see /usr/include/sys/dklabel.h) for all the existing partitions (dkl_nblk != 0) by adding the number of cylinders that the x86 image takes up.
• Recalculate the VTOC checksum
• Write the new VTOC to sector 0 of the new composite image.

Here's a program that performs all the steps listed above:

 

/*****************************************************************************/
/*                                                                           */
/* mkdual.c - Create a single DVD image from both the SPARC and x86 versions */
/*            of SUE that is capable of booting on both types of hardware    */
/*                                                                           */
/*                                                                           */
/* Accomplishing this takes advantage of the fact that there is no partition */
/* structure on the GRUB-based x86 CD/DVD. We can sneak the SPARC VTOC into  */
/* sector 0 on the x86 image since x86 doesn't use this area. The basic      */
/* steps performed by this program are:                                      */
/*                                                                           */
/* - Copy the x86 image                                                      */
/* - Take the size of the x86 image and pad up to the next cylinder boundary */
/* - Read the SPARC image VTOC                                               */
/* - Modify the starting cylinders (dkl_cylno) of the VTOC slices by adding  */
/*   the offset represented by the size of the x86 image + padding           */
/* - Tack on the SPARC image to the copied x86 image.                        */
/* - Calculate the new checksum for the SPARC VTOC                           */
/* - Write the new modified VTOC to sector 0 of the composite image          */
/*                                                                           */
/* This may need to be modified when SPARC newboot hits Solaris 10 (u6)      */
/*                                                                           */
/* J. Cecere                                                                 */
/*                                                                           */
/*****************************************************************************/

#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/vtoc.h>
#include <stdlib.h>
#include <sys/utsname.h>
#include <sys/systeminfo.h>
#include <strings.h>

#define ISOCYL 327680           /* Size of a cylinder on an ISO 9660 fs */
#define BUFSIZE 131072          /* Read/write buffer size               */
#define PTYPE "sparc"       /* Type of machine this can be run on   */

struct dk_label get_vtoc(char *dev) {

        /* Read in a VTOC from the given device. Exit if it's not a VTOC */

        int fd,a;
        struct dk_label vt;

        if ((fd=open(dev, O_RDONLY)) == -1) {
                perror(dev);
                exit(1);
        }

        if ((a = read (fd, (void *)&vt, sizeof (struct dk_label))) == -1) {
                perror("read:");
                exit(a);
        }

        close (fd);

        if (vt.dkl_vtoc.v_sanity != VTOC_SANE) {
                fprintf(stderr,"No VTOC found\n");
                exit(1);
        }

        return(vt);
}

int main(int argc, char *argv[]) {

        char *x86image,*sparcimage,*outfile;
        int a,k;
        int ifd,ofd;
        struct dk_label vt;
        unsigned short *ckptr;
        unsigned short cksum;
        char buf[BUFSIZE];
        struct stat ibuf;
        long long total;
        int percent,oldpercent;
        int numcyl,pad;
        char *padbuf;
        char ptype[SYS_NMLN];

        /* First check that we're running on SPARC */

        sysinfo(SI_ARCHITECTURE, ptype, sizeof(ptype));

        if (strcmp(ptype,PTYPE) != 0) {
                fprintf(stderr,"%s must be run on SPARC\n",*argv);
                exit(1);
        }

        /* Usage requires three files as arguments */

        if (argc != 4) {
                printf("Usage: %s <x86 image> <sparc image> <outfile>\n",*argv);
                exit(0);
        }

        x86image   = *(argv+1);
        sparcimage = *(argv+2);
        outfile    = *(argv+3);

        /* Get the VTOC from the SPARC image */

        vt = get_vtoc(sparcimage);

        /* Copy the x86 image */

        if ((ifd=open(x86image, O_RDONLY)) == -1) {
                perror(x86image);
                exit(1);
        }

        /* Get the file size */

        stat(x86image,&ibuf);

        /* Open the output file with the correct mode */

        if ((ofd=open(outfile,O_RDWR | O_CREAT | O_TRUNC, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH)) == -1) {
                perror("outfile");
                exit(1);
        }
        
        /* total, percent, and oldpercent are used to print out the */
        /* percentage complete while the copy is being performed    */

        total=0;
        percent=0;
        oldpercent=-1;

        /* Copy the ISO file into the image file */

        while ((a=read(ifd,&buf,sizeof(buf))) != 0) {
                if (a == -1) {
                        perror(x86image);
                        exit(-1);
                }

                total+=a;
                percent=(total*100)/ibuf.st_size;
                if (percent != oldpercent) {
                        printf("\rCopying in x86 image...%d%%",percent);
                        fflush(stdout);
                        oldpercent=percent;
                }

                if ((write(ofd,&buf,a)) == -1) {
                        perror(outfile);
                        exit(-1);
                }
        }

        close(ifd);

        printf("\rCopying in x86 image...done.\n");

        printf("Calculating new offsets and padding...");

        /* numcyl is the number of cylinders in the x86 image, before figuring
           out if we need padding. */

        numcyl = ibuf.st_size/ISOCYL;

        /* Calculate the number of pad bytes we need to have the end of the x86
           image fall on a cylinder boundary */

        pad = ISOCYL - (ibuf.st_size%ISOCYL);

        /* If our pad is our cylinder size or 0, then we don't need padding as
           our image already falls on a cylinder boundary */

        if (pad != 0 && pad != ISOCYL) {

                /* We need padding. Add one to the number of cylinders since we're 
                   padding up. Then create a padding buffer. */

                numcyl++;
                padbuf = malloc(pad);

                /* Ok, so we didn't put anything in padbuf, so whatever garbage
                   happens to be in memory will be in it. However, we don't care
                   since this area of the image will never be used for anything.
                   Write the padding buffer out. */

                write(ofd,padbuf,pad);
                free(padbuf);
        }

        /* Now we take the starting cylinders for all our slices and add the
           offset of the size of the x86 image (which will be in whole cylinders */

        for (k=0;k<NDKMAP;k++) {

                /* If this slice wasn't defined, don't bother */

                if (vt.dkl_map[k].dkl_nblk != 0) {

                        /* Recalculate the cylinder offsets for the slices. */

                        vt.dkl_map[k].dkl_cylno += numcyl;

                }
        }
        printf("done\n");

        printf("Calculating new VTOC checksum...");
        cksum = 0;
        ckptr = (unsigned short *)&vt;
        for (k=0; k<255; k++) {
                cksum ^= *ckptr;
                ckptr++;
        }

        vt.dkl_cksum = cksum;
        printf("done\n");

        /* Get the size of the SPARC image so that we can print an accurate
           percentage count while we're copying it. */

        stat(sparcimage,&ibuf);

        if ((ifd=open(sparcimage, O_RDONLY)) == -1) {
                perror(sparcimage);
                exit(1);
        }

        /* ofd is still open and pointing to the end of the outfile. Start
           appending the SPARC image to it */

        total=0;
        percent=0;
        oldpercent=-1;

        /* Copy the ISO file into the image file */

        while ((a=read(ifd,&buf,sizeof(buf))) != 0) {
                if (a == -1) {
                        perror(sparcimage);
                        exit(-1);
                }

                total+=a;
                percent=(total*100)/ibuf.st_size;
                if (percent != oldpercent) {
                        printf("\rAppending SPARC image...%d%%",percent);
                        fflush(stdout);
                        oldpercent=percent;
                }

                if ((write(ofd,&buf,a)) == -1) {
                        perror(outfile);
                        exit(-1);
                }
        }

        close(ifd);
        printf("\rAppending SPARC image...done.\n");

        /* Now rewrite the VTOC with the updated information */

        printf("Writing new VTOC...");
        lseek(ofd,0,SEEK_SET);
        write(ofd, (void *)&vt, sizeof(struct dk_label));
        close(ofd);
        printf("done\n");

}

 

(2008-03-21 08:44:16.0) Permalink Comments [1]

New Blueprint article

I've published a new Blueprint article entitled

GRUB and the Solaris Operating System on x86 Platforms - A Guide to Creating a Customized Boot DVD

This article is a followup to the one Dana Fagerstrom and I published last year called:

Creating a Customized Boot CD/DVD for the Solaris Operating System for x86 Platforms

The difference between the two is that the first one documents how to customize a DVD with versions of Solaris that use the Device Configuration Assistant as the boot loader. Since Solaris (as of S10 01/06) now uses GRUB, the procedure has completely changed. If you're interested, here are the links to the two articles:

DCA based Solaris
GRUB based Solaris



(2006-09-01 08:34:16.0) Permalink Comments [1]

Building 64-bit Open Source applications in Solaris - Part 4 Well, it's been a while since I've had a chance to get back to this. The last two components to build are MySQL and PHP. Today, we'll visit MySQL. There are a lot of issues to be aware of when building MySQL with gcc in Solaris. I'll document them here, but I would recommend using Sun Studio 11 instead. If you do use gcc, make sure that you read the notes below before proceeding.



 

---

MySQL

---

Version: 5.0.21
Dependencies: OpenSSL, (Berkeley DB - included)
Download from: http://dev.mysql.com/downloads/mysql/5.0.html
Build information:

gcc

---

CC="gcc -m64 -O3"
CXX="g++ -m64 -O3"
CPPFLAGS="-I/usr/local/ssl/include"
LDFLAGS="-L/usr/local/ssl/lib -R/usr/local/ssl/lib -L/usr/sfw/lib/`isainfo -k` -
R/usr/sfw/lib/`isainfo -k`"
export CC CXX CPPFLAGS LDFLAGS
./configure --enable-local-infile --with-openssl=/usr/local/ssl --with-berkeley-
db --enable-thread-safe-client
make
make install

gcc + static libgcc

---

CC="gcc -m64 -static-libgcc"
CXX="g++ -m64 -static-libgcc"
CPPFLAGS="-I/usr/local/ssl/include"
LDFLAGS="-L/usr/local/ssl/lib -R/usr/local/ssl/lib -R/usr/sfw/lib/`isainfo -k` -
L/tmp/lib"
export CC CXX CPPFLAGS LDFLAGS
mkdir -p /tmp/lib
ln -s `g++ -m64 -print-file-name=libstdc++.a` /tmp/lib
./configure --enable-local-infile --with-openssl=/usr/local/ssl --with-berkeley-
db --enable-thread-safe-client
make
make install

SunStudio 11

---

arch=`isainfo -k | sed s/sparc//`
CC="/opt/SUNWspro/bin/cc -fast -xarch=${arch}"
CXX="/opt/SUNWspro/bin/CC -fast -xarch=${arch}"
CPPFLAGS="-I/usr/local/ssl/include"
LDFLAGS="-L/usr/local/ssl/lib -R/usr/local/ssl/lib"
export CC CXX CPPFLAGS LDFLAGS
./configure --enable-local-infile --with-openssl=/usr/local/ssl --with-berkeley-
db --enable-thread-safe-client
make
make install

---

Configuration

This is just a generic procedure on how to get the database up and running after installing it. Tweaking and configuring MySQL is outside the scope of what's being described here.

• Add a mysql user and a db group to your system (useradd/groupadd)
• chown mysql:db /usr/local/var
• /usr/local/bin/mysql_install_db --user=mysql  (set up initial databases/tables)
  
• /usr/local/bin/mysqld_safe --user=mysql &  (start daemon up in 'safe' mode)
  
• /usr/local/bin/mysqladmin -u root password "password"  (set the db password)
• pkill mysqld

Now start the database with

/usr/local/share/mysql/mysql.server start

You should now be able to access the database:

mysql -uroot -ppassword

---

Notes:

• SPARC gcc ONLY. Due to a bug in the version of gas (GNU assembler0 in SUNWbinutils included with Solaris 10, MySQL may not build iN 64-bit mode and can fail with the following error:

•  For Solaris 10u1, bug 6220191 might cause the build to fail with the following message (<arch> = amd64 or sparcv9):

• For s10u1 and snv_40 (and below), bug 6430820 might cause the build to fail with the following error message:

• MySQL has Berkeley DB and zlib bundled in with it. It's a lot easier to use the bundled in versions than try to build with whatever versions you have on your system.
•  We specify the -m64 option in CC and CXX because it doesn't migrate this option to everywhere it's needed if we use CFLAGS and CXXFLAGS.
• Since MySQL has parts that are written in C++, gcc will build in a dependency to libstdc++ (along with the libgcc dependency). The static libgcc procedure here makes a symbolic link of libstdc++ to a temporary directory that we create. We then add this temporary directory to LDFLAGS so that it gets statically linked at build time. This effectively removes the dependency from the created binaries.
• The MySQL build process may get confused if you try to run a parallelized make such as gmake -j or dmake -j. Run a serialized make. One of the main issues is that MySQL has it's own copy of the source for Berkeley DB, which needs to be built before anything can link to it. A parallelized make most likely won't build Berkeley DB before the rest of the source.

We'll document building PHP in the next entry.


(2006-09-01 08:15:19.0) Permalink Comments [4]

Building 64-bit Open Source applications in Solaris - Part 3 As I mentioned in the previous entry, I had some issues with running Apache 2.2 as a secure web server when compiled with Sun Studio 11. The current workaround is to just use the '-xO5'  optimization instead of '-fast' (which is a macro that incorporates a whole slew of optimization parameters).

Because of this, I'll include the instructions for setting up both Apache 2.0.58 and Apache 2.2.2. The build process will be identical, except for what I mentioned above. The configuration is a little different as things were shuffled around from 2.0 to 2.2.

 

---

Apache Web Server

---

Version: 2.0.58, 2.2.2
Dependencies: Berkeley DB, OpenSSL
Download from: http://httpd.apache.org/
Build information:

gcc

---

CFLAGS="-m64 -O3"
CPPFLAGS="-I/usr/local/ssl/include"
LDFLAGS="-L/usr/local/ssl/lib -R/usr/local/ssl/lib -R/usr/sfw/lib/`isainfo -k`"
export CFLAGS CPPFLAGS LDFLAGS
./configure --enable-so --with-ssl=/usr/local/ssl --enable-ssl=shared --with-mpm=worker
make
make install

gcc + static libgcc

---

CC="gcc -static-libgcc"
CFLAGS="-m64 -O3"
CPPFLAGS="-I/usr/local/ssl/include"
LDFLAGS="-L/usr/local/ssl/lib -R/usr/local/ssl/lib"
export CC CFLAGS CPPFLAGS LDFLAGS
./configure --enable-so --with-ssl=/usr/local/ssl --enable-ssl=shared --with-mpm=worker
make
make install

SunStudio 11

---

CC="/opt/SUNWspro/bin/cc"
test `isainfo -k` = amd64 && opt="-fast" || opt="-xO5"
CFLAGS="-fast -xarch=`isainfo -k | sed s/sparc//`" (for 2.0.58)
CFLAGS="-xO5 -xarch=`isainfo -k | sed s/sparc//`" (for 2.2.2)
CPPFLAGS="-I/usr/local/ssl/include"
LDFLAGS="-L/usr/local/ssl/lib -R/usr/local/ssl/lib"
export CC CFLAGS CPPFLAGS LDFLAGS
./configure --enable-so --with-ssl=/usr/local/ssl --enable-ssl=shared --with-mpm=worker
make
make install

---

Configuration for 2.0.58:

In /usr/local/apache2/conf/httpd.conf:

• Change Group #-1 to Group nobody
• Set ServerName to the fully qualified name (in DNS) of the system
  
• Set ServerAdmin to an appropriate email address
• Set UseCanonicalName to On

In /usr/local/apache2/conf/ssl.conf:

• Set _default_ in <VirtualHost _default_:443> to the same as you set ServerName in httpd.conf (keep the :443). For example, <VirtualHost myserver.sun.com:443>
  
• Set ServerName to the same as you set in httpd.conf (with :443 appended)
• Set ServerAdmin to the same as you set in httpd.conf
• Set SSLCertificateFile to /usr/local/apache2/conf/server.crt
• Set SSLCertificateKeyFile to /usr/local/apache2/conf/server.key

Now we need to create the key and certificate for https. We do this using openssl.

• PATH=/usr/local/ssl/bin:$PATH
• cd /usr/local/apache2/conf
  
• openssl genrsa -out server.key 2048
• openssl req -new -x509 -key server.key -out server.crt -days 365

The last command will create a certificate that will expire in 365 days. Answer the questions from the command appropriately. For the question:


enter the value that you used for ServerName in httpd.conf and ssl.conf.

Once you're done, start apache via:

---

Configuration for 2.2.2:

In /usr/local/apache2/conf/httpd.conf:

• Set ServerName to the fully qualified name (in DNS) of the system
  
• Set ServerAdmin to an appropriate email address
• Uncomment Include conf/extra/httpd-mpm.conf
• Uncomment Include conf/extra/httpd-default.conf
• Uncomment Include conf/extra/httpd-ssl.conf

In /usr/local/apache2/conf/extra/httpd-default.conf

• Set UseCanonicalName to On

In /usr/local/apache2/conf/extra/httpd-ssl.conf

• Set _default_ in <VirtualHost _default_:443> to the same as you set ServerName in httpd.conf (keep the :443). For example, <VirtualHost myserver.sun.com:443>
• Set ServerName to the same as you set in httpd.conf (with :443 appended)
• Set ServerAdmin to the same as you set in httpd.conf

Now we need to create the key and certificate for https. We do this using openssl.

• PATH=/usr/local/ssl/bin:$PATH
• cd /usr/local/apache2/conf
  
• openssl genrsa -out server.key 2048
• openssl req -new -x509 -key server.key -out server.crt -days 365

The last command will create a certificate that will expire in 365 days. Answer the questions from the command appropriately. For the question:


enter the value that you used for ServerName in httpd.conf and ssl.conf.

Once you're done, start apache via:

/usr/local/apache2/bin/apachectl start

---

Notes:

• Once you're done with this configuration, you'll be able to access your new web server via https.
• PHP will require more configuration changes to Apache, but we'll cover that when we go through the PHP build process.
  

Next entry will be on building MySQL.

(2006-06-21 13:32:13.0) Permalink Comments [0]

Building 64-bit Open Source applications in Solaris - Part 2 I've had a very specific problem come up with Apache 2.2.2 and SunStudio 11 when building it with the -fast option. The httpd process will core dump 9 times out of 10 when accessed via https. I've filed a bug against SunStudio, but until this gets resolved we'll have to just use -xO5 optimization. If this doesn't mean anything to you, don't worry about. I'll mention more on it later when we actually get to building Apache.

Now on to building something. All the build procedures I present here assume that you're starting out in the top level source directory. We start with the prerequisite software to Apache, MySQL, and PHP. For our purposes, this will be Berkeley DB, libxml2, OpenSSL, and OpenLDAP.

---

Berkeley DB

---

Version: 4.4.20
Dependencies: None
Download from: http://dev.sleepycat.com/
Build information:

gcc

---

CC="gcc -m64"
CFLAGS="-O3"
LDFLAGS="-L/usr/sfw/lib/`isainfo -k` -R/usr/sfw/lib/`isainfo -k`"
export CC CFLAGS LDFLAGS
cd build_unix
../dist/configure
make
make install

gcc + static libgcc

---

CC="gcc -m64 -static-libgcc"
CFLAGS="-O3"
cd build_unix
../dist/configure
make
make install

SunStudio 11

---

CC="/opt/SUNWspro/bin/cc"
CFLAGS="-fast -xarch=`isainfo -k | sed s/sparc//`"
export CC CFLAGS
cd build_unix
../dist/configure
make
make install


Note that I use a little shell scripting in the setting of CFLAGS (-xarch=`isainfo -k | sed s/sparc//`). The reason for this is to make the build process identical on SPARC systems and AMD64 (x86) systems. For a SPARC system, -xarch=`isainfo -k | sed s/sparc//` will resolve to -xarch=v9 and on AMD64 systems it will resolve to -xarch=amd64. These are the respective options to build 64-bit binaries on the two platforms. I'll use this type of thing throughout the various build procedures.

---

OpenSSL

---

Version: 0.9.8b
Dependencies: None
Download from: http://www.openssl.org/source/
Build information:

gcc

---

./Configure shared <type> -R/usr/sfw/lib/`isainfo -k` -R/usr/local/ssl/lib
make
make install

gcc + static libgcc

---

./Configure shared <type> -static-libgcc -R/usr/local/ssl/lib
make
make install

SunStudio 11

---

./Configure shared <type> -R/usr/local/ssl/lib
make
make install

<type> is equal to one of the following:

	SPARC	AMD64
gcc	solaris64-sparcv9-gcc	solaris64-x86_64-gcc
SunStudio	solaris64-sparcv9-cc	solaris64-x86_64-cc

Notes:

• OpenSSL is included in Solaris 10u1 in /usr/sfw with both 32 and 64-bit libraries. However, only the 32-bit executable is present. The version included in Solaris 10u1 is 0.9.7 and might work as well for all the software listed here.
• For SunStudio 11, if you fail to apply patch 120760-05 (SPARC) or 120759-05 (i386), the build will complete successfully, but the resulting code will be flawed. This will be evident is you run 'make test' after building OpenSSL. The AES-128-ECB cypher test will fail. Additionally, you will not be able to create a usable certificate for Apache for https.
  

---

libxml2

---

Version: 2.6.23
Dependencies: None
Download from: ftp://xmlsoft.org/libxml2/
Build information:

gcc

---

CFLAGS="-m64 -O3"
LDFLAGS="-R/usr/sfw/lib/`isainfo -k`"
export CFLAGS LDFLAGS
./configure --without-python
make
make install

gcc + static libgcc

---

CC="gcc -m64 -O3 -static-libgcc"
export CC
./configure --without-python
make
make install

SunStudio 11

---

CC="/opt/SUNWspro/bin/cc"
CFLAGS="-fast -xarch=`isainfo -k | sed s/sparc//`"
export CC CFLAGS
./configure --without-python
make
make install

Notes:

• Version 2.6.11 or higher is required for PHP 5.1.4. Unfortunately, Solaris 10U1 only has 2.6.10, so this is required if you want XML support in PHP.
• Building in python support is a separate project that requires other dependencies. In Solaris 10, python is in a non-standard place (/usr/sfw) so the libxml configure won't find it by default, so you won't need to specify --without-python. However, in Solars 11 (Nevada, OpenSolaris, etc.), python has been moved to /usr/bin so the configure will find it and try to use it if --without-python isn't specified.
  
  
  

---

OpenLDAP

---

Version: 2.3.23
Dependencies: Berkeley DB, OpenSSL
Download from: http://www.openldap.org/software/download/
Build information:

gcc

---

CC="gcc -m64 -O3 -D_AVL_H"
CPPFLAGS="-I/usr/local/ssl/include -I/usr/local/BerkeleyDB.4.4/include"
LDFLAGS="-L/usr/local/ssl/lib -R/usr/local/ssl/lib -L/usr/local/BerkeleyDB.4.4/lib -R/usr/local/BerkeleyDB.4.4/lib -R/usr/sfw/lib/`isainfo -k`"
export CC CPPFLAGS LDFLAGS
./configure
make depend
make
make install

gcc + static libgcc

---

CC="gcc -m64 -O3 -static-libgcc -D_AVL_H"
CPPFLAGS="-I/usr/local/ssl/include -I/usr/local/BerkeleyDB.4.4/include"
LDFLAGS="-L/usr/local/ssl/lib -R/usr/local/ssl/lib -L/usr/local/BerkeleyDB.4.4/lib -R/usr/local/BerkeleyDB.4.4/lib"
export CC CPPFLAGS LDFLAGS
./configure
make depend
make
make install

SunStudio 11

---

CC="/opt/SUNWspro/bin/cc"
test `isainfo -k` != amd64 && opt="-fast "
CFLAGS="${opt}-xarch=`isainfo -k | sed s/sparc//` -D_AVL_H"
CPPFLAGS="-I/usr/local/ssl/include -I/usr/local/BerkeleyDB.4.4/include"
LDFLAGS="-L/usr/local/ssl/lib -R/usr/local/ssl/lib -L/usr/local/BerkeleyDB.4.4/lib -R/usr/local/BerkeleyDB.4.4/lib"
export CC CFLAGS CPPFLAGS LDFLAGS
./configure
make depend
make
make install

Notes:

• Due to bugs 6426797 and 6423673, OpenLDAP can not currently be compiled with any optimizations on the AMD64 platform with SunStudio 11 in 64-bit mode. This doesn't affect SPARC.
• The '-D_AVL_H' in CFLAGS is necessary for Solaris 10U2 and above. Apparently, avl.h was added to Solaris in kernel patch 118833-17 and is incompatible with OpenLDAP (which has it's own version of this file). This CFLAG option isn't necessary for prior versions of the kernel patch, but it won't hurt anything either. See http://www.openldap.org/its/index.cgi/Build?id=4607
  
  
  

---

Next week we'll start going through the build processes of Apache, MySQL, and PHP

(2006-06-16 11:54:08.0) Permalink Comments [2]

Building 64-bit Open Source applications in Solaris - Part 1

Mads Toftum suggests that we use Apache 2.2 (latest) rather than the older 2.0. Personally, I have yet to put 2.2 into production anywhere. There are always two perspectives when it comes to new versions of software. The developer and the end-user. The developer has a lot of insight into what's been fixed in the newer version and, knowing this, can't imagine why anyone would want to run the older version with all it's bugs and missing features. The end-user, on the other hand, sees an application that works fine for his use, and is wary to upgrade in fear that doing so will break something or lead him down a long path of upgrading dependent software. There are valid points to be taken on both sides. Usually the developer is right about getting the bug fixes and features in, but developers can't test for every configuration, and the end-user might end up with something broken that wasn't broken before.

Be that as it may, since we're starting from scratch, it's probably prudent to just use the latest that's out there. I've changed the version listed in my prior entry from Apache 2.0 to 2.2.

Mads also suggests that we use the worker MPM (multi process model) rather than the default prefork in building apache. This is an excellent suggestion and I agree with this. It takes advantage of the inherent multi-threaded design of the Solaris kernel. In order to do this, we'll need to build PHP with the tsrm-pthreads option (tsrm = thread safe resource manager) as well.

The Basics

Let's go through some basics about building open source software. The standard process for building most open source software is simple:

• ./configure

• make

• make install

This is fine if you want to take the default feature set and configuration of whatever you're building and everything on your system is where the application thinks it should be. Unfortunately, this is rarely the case. The configure script has the purpose of figuring out the characteristics of your particular machine and creating makefiles accordingly. However, most of the time it needs a little help in figuring out where things are. As we'll see, there are a number of command line options that need to be added. In addition, there are also environmental variables that affect how configure will write the makefiles and other configuration information. The variables that we'll be using are:

• CC - Defines what the C compiler is

• CXX Defines what the C++ compiler is

• CFLAGS - Arguments to pass to the C compiler

• CXXFLAGS - Arguments to pass to the C++ compiler

• CPPFLAGS - Arguments to pass to the C preprocessor. Mostly used to define where to look for extra header (*.h) files via -I<header file directory>

• LDFLAGS - Arguments to pass to the linker. Used to add extra directories to look for libraries in. Normally, there are two FLAGS to pass for each library directory. -L defines where to look for libraries at compile (link) time. -R defines where to look for libraries at run time. There is a distinction between the two. If you neglect to add a needed directory via the -L option, the build will most likely fail with a 'missing symbols' error. On the other hand, if you neglect to add a needed directory via the -R option, then you will need to set LD_LIBRARY_PATH to the directory(s) you left out or your program won't be able to find the library(s) it needs when you try to run it.

We will assume that we're using the Bourne shell (sh) or the Korn shell (ksh). Because of this, these variables will need to be exported after being set before we run ./configure.

There are other variables that affect the build process of various software, but these are all the ones that we'll be using.

More than one way to skin a cat...

In this series, I will present three ways to build all the software here. They are:

gcc

The gcc version that I used is 3.4.3 and is distributed with Solaris 10 in the SUNWgcc package and resides in /usr/sfw. This version uses the GNU assembler (gas 2.15) by default. I'll discuss the significance of this when we build MySQL. By default, gcc will leave a dependency on libgcc. This means that the -L and -R flags will need to point to the non-standard location of the 64-bit libgcc_s, which is in /usr/sfw/lib/`isainfo -k` (isainfo -k will resolve to either sparcv9 or amd64).

gcc from some other source (SunFreeware, Blastwave, etc.) should work as well.

gcc + static libgcc

If you don't want to rely on the assumption that libgcc will be present on the system that you intend to install your software on (assuming it's a different machine than the one you're building it on), then choose this option if you want to use gcc to build with. The end result for all the software built with this option will have no dependencies on libgcc or libstdc++ (the C++ library - only used by MySQL here).  This assumes that you built all a software package's dependencies this way as well. If not, then you may still end up with dependencies on libgcc.

SunStudio 11

This is Sun's development kit. It contains C, C++, and Fortran compilers, an integrated development environment (IDE), and a really cool debugger (dbx). It's available for free from this site:

http://developers.sun.com/prodtech/cc/downloads/index.jsp

If you've already registered on any of Sun's web sites, you should be able to use the same login and password here.

If you choose to use SunStudio, please install the latest version of patch 120760 (sparc) or 120759 (x86) after installing it. If you don't, OpenSSL will not build correctly.

That's it for now. Next week we'll dig into the actual builds.

(2006-06-09 11:20:49.0) Permalink

Building 64-bit Open Source applications in Solaris - Intro

I started building open source applications from source in Solaris (actually it was SunOS 4.x at the time) almost 15 years ago in my previous life as a system administrator. In that life, I supported a lot of Sun client workstations.  Open source was a great way to add new and improved functionality to the network, so learning to build them was a must.

Open source is still a great way to add functionality to a network. In this series of entries, I'll be documenting specifics on building the popular Apache/MySQL/PHP combination as 64-bit applications in Solaris for both the SPARC and AMD64 platforms. The end result is that you'll be able to download the source code from 7 open source applications and put them together to create a secure web application server. Below is a list of applications, versions, and dependencies, as well as where to download them from.

Application name	Version	Dependent on	Download from
Berkeley DB	4.4.20	n/a	http://dev.sleepycat.com/
OpenSSL	0.9.8b	n/a	http://www.openssl.org/source/
libxml2	2.6.23	n/a	ftp://xmlsoft.org/libxml2/
OpenLDAP	2.3.23	Berkeley DB, OpenSSL	http://www.openldap.org/software/download/
Apache Web Server	2.2.2	Berkeley DB, OpenSSL	http://httpd.apache.org/
MySQL	5.0.21	Berkeley DB (included)	http://dev.mysql.com/downloads/mysql/5.0.html
PHP	5.1.4	Berkeley DB, OpenSSL, OpenLDAP, MySQL, Apache	http://www.php.net/downloads.php

The dependencies listed here are essentially for the context of what we're going to accomplish here, and by no means comprehensive lists of features available to these applications.

Tomorrow we'll look at the build environment.

(2006-06-08 20:01:02.0) Permalink Comments [1]