Welcome to Nevada! Nevada is our code-name for the next version of Solaris. For now at least, the uname -r output is 5.10.1 although that is subject to change. As I did for the last couple of Solaris 10 SX builds, I'll attempt to keep you abreast of the changes happening in each SX release. I missed doing one which described the delta between SX 11/04 (s10_72) and the FCS build of Solaris 10. The most important of those changes include:

• Inclusion of gcc for SPARC, x86 and AMD64 (/usr/sfw/bin/gcc).
• Intel 10GB NIC driver (ixgb driver)
• svcadm (part of SMF) picked up a synchronous mode (via -s)
• BIND 9 became the default name server. BIND 8 was removed.
• A large fraction of binaries delivered (including kernel modules) are now cryptographically signed. See elfsign(1).

Ok. Now entering Nevada. Notable New Features in Solaris Nevada, Build 7 (AKA Solaris Express 02/2005):

Desktop

• Updated Xorg from 6.8.0 to 6.8.2RC2, including numerous bug fixes and new hardware support (see the X.org release notes). The final version of 6.8.2 will be available in a future Solaris Express build.
• An annoying bug in the /usr/sfw/bin/mozilla prevents it from starting up properly. Edit the OS_VERSION check in the script to work around the problem.
• You can now double-click .jnlp (java web-start) and .jar files to run them under GNOME.

Hardware support

• via823x SADA audio driver on x86 and AMD64 platforms.
• Chelsio 10gb NIC driver available on all platforms (SPARC, x86, AMD64).

Security

• 64-bit openssl(1) command available. Solaris already ships with a 64-bit openssl library. The openssl command provides a tool for using various cryptography functions of OpenSSL's crypto library from the shell.
• Support for a PKCS#11 "MetaSlot". This is an extension to the Cryptographic framework which presents a single slot which is the union of the capabilities of other slots which are loaded in the framework.
• IKE gets a performance boost by using the encryption framework. IKE is also now fully compliant with RFC 3947 (NAT-T support).

Storage

• iSCSI devices are now supported via the new iscsiadm(1m) command.
• The fcinfo(1m) utility is now available; this utility can be used to list fibre channel ports on the system in a concise and clear fashion.

Performance

• Hierarchical (Multi-level) Lgroup support. Solaris has an abstraction called an Lgroup (latency group) which is the way in which the system tracks NUMA system topology. Traditionally, Solaris has run on systems with no difference in latency (traditional SMP systems) or only two levels of latency (local memory and remote memory). Newer system designs have more levels. For example, 4-CPU Opteron systems have 3 such levels; 8-way Opteron systems may have 4 levels. This project enables better performance on ring and ladder system topologies, and picks up performance wins on Oracle (TPC-SO), Fluent, and other benchmarks. There are some new liblgrp APIs to go along with this work (lgrp_latency_cookie(3LGRP)).
• Faster memmove(3c) (anywhere from 0-400%, 40% is typical) on 32-bit x86 platforms. AMD64 performance of memmove(3c) and bcopy(3c) were also improved.
• Improved context switch performance on AMD64.
• Much improved performance on 32-bit x86 string functions: strcpy(3c) (as much as 50%), strlen(3c) (as much as 25% on long strings) and strchr(3c) (as much as 45% on long strings).

Other:

• New TCP_INIT_CWND TCP socket option allows the congestion control window calculation to be overridden with a user specified value. See tcp(7p) for full details.

(2005-03-01 21:20:00.0) Permalink Comments [6]
Trackback: http://blogs.sun.com/dp/entry/what_s_new_in_solaris3

I run a web proxy server for folks in the office; we use it as a longterm testbed for Solaris. But, in the insanity leading up to the release of Solaris 10, I've had little time to work on it. Recently I got a new server to host the cache; and so I've been busy putting it together. We've always used Squid as our proxy software. Personally, I have some qualms about Squid's design, but with five years of experience using it, I think we'll probably stick with it for now. It's a curious thing that there doesn't appear to be a significant competitive open source alternative to Squid (the forthcoming Apache 2.1 is moving mod_cache out of "experimental" support so perhaps that will be worth considering?).

Setting aside design complaints, being able to effectively administer Squid is a big priority, so recently I worked on getting it properly under the control of the Service Management Facility (SMF). It's also a good example of how to improve a program's administrative controls with SMF.

The first task was to look through Squid's existing start/stop/restart capabilities. There's a RunCache script, which I had always thought was the supported way to start the daemon. Looking at the documentation, RunCache is now aparently obsolete, but still installed along with squid anyway (sigh). RunCache has many problems which I won't detail here.

In the same neighborhood, there is the squid binary, which has a number of relevant command line options:

Usage: squid [-dhsvzCDFNRVYX] [-f config-file] [-[au] port] [-k signal]
...
       -f file   Use given config-file instead of
                 /aux0/squid/etc/squid.conf
...
       -k reconfigure|rotate|shutdown|interrupt|kill|debug|check|parse
                 Parse configuration file, then send signal to 
                 running copy (except -k parse) and exit.
       -s        Enable logging to syslog.
...
       -z        Create swap directories
...
       -N        No daemon mode.
...

To add to the complexity, squid has its own restarter directly built into itself. This is somewhat suboptimal, as SMF tends to trump these facilities, and allows monitoring software to have visibility into restart events. Anyway, we can make use of the -k option to control the daemon to some degree, and give the administrator the power to create multiple service instances if we use the -f option. In my testing, I found the -k reconfigure option to be somewhat useless, so I decided not to implement an SMF 'refresh' method. Perhaps I missed something?

Another problem we'd like to solve is that Squid doesn't operate properly "out of the box." First, one must run the daemon with the -z option in order to create the cache metadata. I'm not sure why the squid team made this decision; I certainly don't think it's a good one. Our startup scripting can simply take care of cache creation for the administrator. After working out the right set of dependencies for the cache as I'd set it up (./configure --disable-internal-dns --enable-ssl --prefix=/aux0/squid --enable-storeio='ufs aufs'), I prepared a service manifest file which captured those dependencies; the dependencies look like this:

$ svcs -d squid 
STATE          STIME    FMRI
online         Jan_26   svc:/milestone/network:default
online         Jan_26   svc:/system/filesystem/local:default
online         Jan_26   svc:/network/dns/client:default
online         Jan_26   svc:/milestone/sysconfig:default

The network milestone is the stable way to depend on "networking being up on the box." A buglet in some of the S10 FCS manifests (notably, Apache) is that some of them have finer grained, and less stable dependencies (for example, on network/physical). When stable dependencies in the form of milestones are available, please use them.

Note that the default mode for squid is to use it's own internal DNS library (ugh), so you may or may not need the DNS dependency. This is (double ugh) a compile time setting. Regardless, you'll want to have an /etc/resolv.conf file present, and the network/dns/client manifest checks for that.

Next, I worked on revising the startup script to be much more intelligent. To start up the cache, it uses squid's -k parse option to decide whether the configuration file has a valid syntax. If not, it exits with the $SMF_ERR_CONFIG error code, which indicates a configuration problem. Next, it populates the cache directory using squid -z as needed. Finally, it starts up the cache. Every failure logs a clear and detailed log message.

I also added a couple of service properties, which the script uses to set its behavior. Ideally, this will be automatically and correctly generated from the configure script in the future. Just tweak the manifest before importing it. In the example manifest, squid has been configured to be installed into /aux0/squid. You will want to search the file and alter all of the places which reference /aux0/squid, adjusting them for your installation (you can also use svccfg after you import the manifest to make corrections). Here is a draft of the network/http-proxy:squid service manifest; and a draft of the svc-squid startup script. To install:

1. Tweak squid.xml to reflect the Squid installation directory.
2. Copy the svc-squid script to the location reflected by squid.xml.
3. svccfg import /path/to/squid.xml
4. svcadm enable squid

I hope this is helpful! I'd be happy to take suggestions for improvement, and please let me know if you wind up using this successfully.

[Sigh. Sometimes I feel like I'm just too slow to post. Since I started this post a month ago, some of the work Trevor posted obviates mine. While I'm not happy about having multiple similar solutions to a single problem I think this represents a substantial improvement, and it did take quite a while to refine into the current state. It has also been checked and nitpicked by the SMF team, so I'm optimistic that it is roughly correct. One interesting result is that my dependencies are different than the set which Trevor worked out. Determining the right set of dependencies is, at present, a bit of a black art.]
(2005-03-01 04:45:01.0) Permalink Comments [7]
Trackback: http://blogs.sun.com/dp/entry/squid_startup_extreme_makeover_with