Meddling in the Affairs of Wizards
John Beck's Weblog
Archives
« April 2008
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
XML
Search

Links

Today's Page Hits: 77

20080306 Thursday March 06, 2008
OGB Positions & Bio

Philosophy

I believe in open communication and open development.

Employment history

Education

Personal


Mar 06 2008, 01:00:52 PM PST Permalink Comments [1]

20070301 Thursday March 01, 2007
How to set up Sendmail certificates
This entry describes how to set up certificates to enable sendmail to use TLS (transport-layer security). Note that for Solaris 10, the command /usr/sfw/bin/openssl and the configuration file /etc/sfw/openssl/openssl.cnf are provided by the SUNWopenssl-commands and SUNWopensslr packages respectively, which are included in the End User (and greater) package meta-clusters. So if your system is installed with the Core (or lesser) package meta-cluster, you will need to install these two packages.
  1. Set up certificates. Read this section in its entirety before trying any of the steps, as there is an important note at the end of the section regarding certificate expiration. (Note that this section was derived from another page written by Greg Shapiro of Sendmail.)
    To create a certificate authority:
    1# cd /etc/mail
    2# mkdir -p certs/CA
    3# cd certs/CA
    4# mkdir certs crl newcerts private
    5# echo "01" > serial
    6# cp /dev/null index.txt
    7# cp /etc/sfw/openssl/openssl.cnf .
    
    Note that the above source path is for Solaris 10; if anyone happens to be doing the same on say, Mac OS X, the source path is /System/Library/OpenSSL/openssl.cnf; FreeBSD 4.11 puts the file in /etc/ssl/openssl.cnf.
    8# vi openssl.cnf
    
    Set values: I changed the dir value from /etc/sfw/openssl to /etc/mail/certs/CA and the stateOrProvinceName_default value from Some-State to California; the former was important but the latter was not.)
    9# openssl req -new -x509 -keyout private/cakey.pem -out cacert.pem -days 365 -c
    onfig openssl.cnf
    
    Notes on how to answer the questions: for Organization Name, I answer Sun Microsystems, for Organizational Unit Name, I answer Solaris, for Common Name, I answer with the fully-qualified host-name (FQHN) of the machine in question (see check-hostname(1M) for help on this), and for Email Address, I answer with my e-mail address.
    To make a new certificate:
    10# openssl req -nodes -new -x509 -keyout newreq.pem -out newreq.pem -days 365 -
    config openssl.cnf
    
    (The certificate and private key are in the file newreq.pem.) To sign the new certificate with the certificate authority:
    11# openssl x509 -x509toreq -in newreq.pem -signkey newreq.pem -out tmp.pem
    12# openssl ca -config openssl.cnf -policy policy_anything -days 365 -out newcer
    t.pem -infiles tmp.pem
    13# rm -f tmp.pem
    
    (The file newcert.pem contains the signed certificate; newreq.pem still contains the unsigned certificate and private key.)

    Note that commands 9, 10 and 12 have a -days 365 argument, which specifies how long until the certificate expires. There does not appear to be a reliable way to extend certificates, and going back to every machine and reapplying this process a year later is a major pain, as I found out the hard way. So the second time thru, I calculated how many days it would be until January 18, 2038, and used that value. :-)

  2. Tell sendmail.cf about the certificates. (Note that this section was derived from another page written by Claus Aßmann, also of Sendmail.) Add:
    define(`confCACERT_PATH', `/etc/mail/certs')dnl
    define(`confCACERT', `/etc/mail/certs/CAcert.pem')dnl
    define(`confSERVER_CERT', `/etc/mail/certs/MYcert.pem')dnl
    define(`confSERVER_KEY', `/etc/mail/certs/MYkey.pem')dnl
    define(`confCLIENT_CERT', `/etc/mail/certs/MYcert.pem')dnl
    define(`confCLIENT_KEY', `/etc/mail/certs/MYkey.pem')dnl
    
    to your .mc file and rebuild your sendmail.cf file and install it in /etc/mail.
  3. General sym-links:
    # cd /etc/mail/certs
    # ln -s CA/cacert.pem CAcert.pem
    # ln -s CA/newcert.pem MYcert.pem
    # ln -s CA/newreq.pem MYkey.pem
    
  4. Fix permissions:
    # chmod go-r MYkey.pem
    
  5. Specific sym-link (note that this section was also derived from Claus' STARTTLS page):
    # C=CAcert.pem
    # ln -s $C `openssl x509 -noout -hash < $C`.0
    
  6. Install other host(s) certificates: For any hosts you wish to exchange secure mail with, grab that host's copy of /etc/mail/certs/CAcert.pem (or whatever file sendmail.cf's CACertFile option points to on that host) and copy it to /etc/mail/certs/host.domain.cert.pem on your host (where host.domain is the other host's FQHN), and repeat step 5, except using C=host.domain.cert.pem.
By doing the above, I am able to send mail between hosts which are both running a version of sendmail compiled with STARTTLS defined and linked with both libssl and libcrypto, and the Received: header shows e.g.:
Received: from jurassic.eng.sun.com ([IPv6:2002:8192:56bb:9259::8192:5932])
        by opal.eng.sun.com (8.13.4+Sun/8.13.4) with ESMTP id j2TNUB8i242496
        (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
        for <jbeck@opal.eng.sun.com>; Tue, 29 Mar 2005 15:30:11 -0800 (PST
)
Received: from opal.eng.sun.com (opal.SFBay.Sun.COM [129.146.89.54])
        by jurassic.eng.sun.com (8.13.4+Sun/8.13.4) with ESMTP id j2TNU7cl571102
        (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK)
        for <jbeck@jurassic.eng.sun.com>; Tue, 29 Mar 2005 15:30:07 -0800 
(PST)
where the verify=OK is the key part (see Claus' STARTTLS page for an explanation of what the various verify= values correspond to).

Mar 01 2007, 08:52:29 PM PST Permalink Comments [1]

20060831 Thursday August 31, 2006
using ZFS to migrate data across boxes
My old desktop (with SCSI disks) died, and I'd been limping along on a loaner until the new one (with SATA disks) arrived a few days ago. The steps needed to migrate the data from one to the other turned out to be remarkably simple, so I thought I would share them in case anyone else ever runs into this. The old box had a single pool:
NAME                    SIZE    USED   AVAIL    CAP  HEALTH     ALTROOT
space                    68G   16.6G   51.4G    24%  ONLINE     -
with two file-systems therein:
NAME                   USED  AVAIL  REFER  MOUNTPOINT
space                 16.6G  50.3G    99K  /export/home
space/jbeck           12.2G  50.3G  12.2G  /export/home/jbeck
space/local           4.25G  50.3G  4.25G  /usr/local
First, on the new box, I ran:
% zpool create space mirror c2d0 c3d0
Then:
% zfs snapshot -r space@today
% zfs send space/jbeck@today | ssh newbox zfs recv -d space
% zfs send space/local@today | ssh newbox zfs recv -d space
The two send/recv runs took roughly 30 and 10 minutes respectively, much less than the almost two hours that the back-to-back tar method used to take to do this. Finally, on the new box, I ran:
% zfs set mountpoint=/export/home space
% zfs set mountpoint=/usr/local space/local
% zfs set sharenfs=on space/jbeck space/local
And that was it! I'm a happy ZFS user, but note that Matt told me that he is working on an RFE to make the above even simpler:
I'm working on some enhancements to zfs send/recv that will simplify this even further, especially in cases where you have many filesystems, snapshots, or changed properties. In particular, you'll be able to simply do:
# zfs snapshot -r space@today
# zfs send -r -b space@today | ssh newbox zfs recv -p -d newpool
The "send -b" flag means to send from the beginning. This will send a full stream of the oldest snapshot, and incrementals up to the named snapshot (eg, from @a to @b, from @b to @c, ... @j to @today). This way your new pool will have all of the snapshots from your old pool.

The "send -r" flag means to do this for all the filesystem's descendants as well (in this case, space/jbeck and space/local).

The "recv -p" flag means to preserve locally set properties (in this case, the mountpoint and sharenfs settings).

For more information, see RFEs 6421959 and 6421958, and watch for a forthcoming formal interface proposal.

Nice work, ZFS team.

Aug 31 2006, 01:58:48 PM PDT Permalink

20060608 Thursday June 08, 2006
Network Auto-Magic
For the last year or so, I've been leading a project called Network Auto-Magic to simplify and automate network configuration on Solaris. See the project page on OpenSolaris for full details about story boards, requirements, architecture, etc., but the exciting news is that we just released a prototype yesterday.

It monitors network interfaces and brings them up and down as availability changes, including the ability to activate user-specified hooks (such as a VPN) automatically. It will scan wireless networks, then offer the user a list of each, including signal strength and encryption type, plus an Other choice to allow the user to select a WLAN which does not broadcast its ESSID. When the user makes a selection, it will remember the selected WLAN, and will connect to it automatically thereafter. It will also prompt for WEP keys as needed and remember them, so such prompting need occur only once. So if you have a recent build of Solaris, try it out and let us know what you think!


Jun 08 2006, 02:34:32 PM PDT Permalink Comments [2]

20051018 Tuesday October 18, 2005
Stand up for what's Right
People a little older than me (I was 3 at the time) will remember the "black power salute", one of the most famous photographs in sports history. In the 1968 Olympic Games in Mexico City, while the national anthem played during the awards ceremony of the men's 200-meter run, gold medalist Tommie Smith and bronze medalist John Carlos stood with their heads bowed, each with a black-gloved fist held straight up. Smith raised his right fist to represent black power, Carlos his left to represent black unity, and both had taken off their shoes and just wore black socks to represent black poverty. The silver medalist, Peter Norman from Australia, wore a pin on his sweat jacket as a show of unity. Smith and Carlos were sent home from the games as a result of their stand, and were harassed and threatened when they got back to America.

Well, that was 1968. As years went by, more and more people saw their stand in a more positive light, understanding that they were saying, in a dignified way, that it was not right for Black Americans to be given equal treatment only when convenient for the country, such as in war or in sports, but treated as second class citizens otherwise. So by 2005, Smith and Carlos tend to be viewed as heroes more than anything else, and heroes overdue for recognition at that.

As it turns out, both Smith and Carlos (and Lee Evans, who won gold in the 400-meter run that year, and participated with them in their planning of "the stand") went to San Jose State University, and a few years ago a student there learned about their story and thought the university should do something to commemorate their stand. It took a few years, but last night (37 years and 1 day after their courageous stand), there was a ceremony on campus at San Jose State, with Smith, Carlos, Norman and Evans all in attendance, where a 20-foot statue was unveiled. I was on hand for the event, and took some pictures.

award ceremony

In the cropped picture above, Carlos is on the left in the bluish suit with the white goatee, Smith is on the right in the greenish suit, Norman is standing behind Carlos' left shoulder, and Evans is standing behind Smith's left shoulder; San Jose Vice Mayor Cindy Chavez is presenting them with an award. In the uncropped version of the picture, you can see how big the (still veiled) statue is.

statue

The statue, seen unveiled, is magnificent. The skin and hair and socks are bronze, while the sweat suits are ceramic tiles. Norman's spot on the dais was intentionally left empty, to make the statue interactive, so people could pose "taking a stand".

I think I was about 7 when I first learned about "the stand", probably during the 1972 Olympic Games in Munich. I remember thinking that they must have been very brave. As I got older, and eventually ran track in high school and college myself, I came to think of them more as heroes, the ultimate in track: winning a gold medal and doing something noble with one's moment of glory. It is hard to put into words how uplifting it was to see these heroes in the flesh, and to hear them speak. Smith and Carlos both credited God for helping them get thru their trying times and both urged the young people present at the ceremony to take a stand and do what's right. Amen, brothers.


Oct 18 2005, 09:41:56 PM PDT Permalink Comments [3]

20050614 Tuesday June 14, 2005
Adding command-line editing/history to an application
As soon as Zones integrated into Solaris 10, we started getting some RFEs for ways to make it better. One of them was to add command history to zonecfg; now that OpenSolaris has been launched, I thought I would share how I did it.

Some existing libraries were suggested for enabling this; after studying them, I recommended libtecla, written by Martin Shepherd, a staff scientist / programmer in the Astronomy department at Caltech, because of its feature set (command-line editing and history, highly customizable, etc.), quality, and license.

Getting libtecla ported to Solaris was trivial, as it already compiled and linked fine. I just had to massage it into the Solaris format, which did not involve any significant code changes, but mainly Solaris Makefiles, lint library stubs, etc. These can be found under usr/src/lib/libtecla for the curious.

The more interesting work was tweaking zonecfg to use libtecla, which included the following steps, which are listed here in a way intended to make it easy to follow for other people wishing to do the same with their application(s):

For those who would like to see all of zonecfg.c in context; it can be found at usr/src/cmd/zonecfg/zonecfg.c.

Technorati Tag:
Technorati Tag:


Jun 14 2005, 08:08:48 AM PDT Permalink Comments [1]

20050126 Wednesday January 26, 2005
building a better spam trap
I've spent a lot of time over the past couple of months trying out some new (and some not so new) anti-spam techniques. Note that this article assumes some familiarity with sendmail m4 macros; see $CFDIR/README for background and all sorts of details on these, where $CFDIR is one of:

These techniques are in the form of FEATURE and HACK m4 macros (the difference being that the former are provided and blessed by sendmail.org / Solaris whereas the latter are not, though a HACK may evolve into a FEATURE in a future release). For a HACK, one would use

        HACK(`hack-name')dnl
in one's .mc file, likewise
        FEATURE(`feature-name')dnl
When installing hacks, one must create $CFDIR/hack (if it does not already exist) and place hack-name.m4 in that directory. Note that the sendmail distribution comes with such a sub-directory but Solaris does not.

Also, to explain some terms used below: the access list is enabled by the FEATURE(`access_db') macro; details on this are in $CFDIR/README, both in its sub-section in the FEATURES section, and in the ANTI-SPAM CONFIGURATION CONTROL section. And FEATURE(`delay_checks') is strongly recommended, as it is needed to enable the overrule by an OK entry in the access list that I mention in a few places; this feature is also described in its subsection in the FEATURES section, as well as in the "Delay all checks" sub-section of the ANTI-SPAM CONFIGURATION CONTROL section.

Anyway, onto the details. In the order I started deploying them:

Overall, spam getting thru my personal domain's mail server to my users (including myself, my wife, my siblings, our mom, etc.) has dropped about 90% since I started using these techniques, despite the ever-increasing spam trends on the rest of the Internet.


Jan 26 2005, 04:16:42 PM PST Permalink Comments [5]

20041010 Sunday October 10, 2004
what is relaying?

I just answered this in comp.unix.solaris for what seems like the millionth time, so I thought I might as well record the answer here for others who might be interested...

People who run mail servers need to be careful not to be an "open relay", or they will get added to various black-lists, and all sorts of mail delivery problems will ensue, as spammers take advantage of open relays when and wherever they find them.

Basically, a message has to either "start here" or "finish here" or it is considered a relay. There is a test and a parameter for each condition. The parameter for "finish here" is the domain part (i.e., right-hand side) of each recipient's e-mail address, and the test is "is that my host name?", though it is often extended to "is that host in my domain?". The parameter for "start here" is the IP address of the client side of the SMTP connection, and the test is "is this the loopback address" (i.e., "is this the local host?"), though that is also often extended to "does this IP address reverse-map to a name in my domain?"

Note that sendmail has disabled relaying by default since version 8.9, which first shipped with Solaris 7, and that the .mc file used to generate the default sendmail.cf on Solaris contains this macro:

DOMAIN(`solaris-generic')
and that domain file contains:
FEATURE(`relay_entire_domain')
which enables the extensions to which I alluded above. Details can be found in the README file, which is in /usr/lib/mail/ on Solaris 9 and earlier, but in /etc/mail/cf/ on Solaris 10; see the ANTI-SPAM CONFIGURATION CONTROL section of that file in particular.

Oct 10 2004, 02:15:51 PM PDT Permalink

20041004 Monday October 04, 2004
no more main vs subsidiary

A common gotcha when setting up a Solaris box on the Internet and trying to send mail has been:

Host unknown (Name server: mailhost.example.com: host not found)
This is because Solaris has for many years shipped two sendmail config files: one for the "real" Internet (main.cf) and one for inside a firewall or elsewhere where DNS may not be configured properly, and average "dumb" hosts need to punt to some "smart" host (subsidiary.cf). As you might guess from the above message, that "smart" host is assumed to have the name mailhost.$m (where $m is the macro for the local domain, "example.com" in the above example) . For backwards compatibility reasons, sendmail.cf has always been a copy of subsidiary.cf rather than main.cf .

Well, starting in Solaris 10 build 63, this problem is fixed. More accurately, the fix is in sendmail version 8.13, which was introduced in s10_63; hopefully, 8.13 will be back-ported to Solaris 9 before too long. I should have thought of this years ago, but better late than never: by inventing a new FallbackSmartHost option, sendmail can handle either scenario in a single configuration:

If specified, the FallBackSmartHost will be used in a last-ditch effort for each host. This is intended to be used by sites with "fake internal DNS", e.g., a company whose DNS accurately reflects the world inside that company's domain but not outside.
This option is specified, with the value mailhost.$m as subsidiary.cf used to use, in the default sendmail.cf, and there is no longer a main.cf or a subsidiary.cf . (Well, technically, they both can still be found, but only as sym-links to sendmail.cf for backwards compatibility.)

Finally, after all these years, a grand unified sendmail configuration! :-)


Oct 04 2004, 09:37:42 PM PDT Permalink

20041001 Friday October 01, 2004
"rm -rf /" protection

Most people who have spent any time on any version of Unix know that "rm -rf /" is about the worst mistake you can make on any given machine. (For novices, "/" is the root directory, and -r means recursive, so rm keeps deleting files until the entire file system is gone, or at least until something like libc is gone after which the system becomes, as we often joke, a warm brick.) Well a couple of years ago one Friday afternoon a bunch of us were exchanging horror stories on this subject, when Bryan asked "why don't we fix rm?" So I did.

The code changes were, no surprise, trivial. The hardest part of the whole thing was that one reviewer wanted /usr/xpg4/bin/rm to be changed as well, and that required a visit to our standards guru. He thought the change made sense, but might technically violate the spec, which only allowed rm to treat "." and ".." as special cases for which it could immediately exit with an error. So I submitted a defect report to the appropriate standards committee, thinking it would be a slam dunk.

Well, some of these standards committee members either like making convoluted arguments or just don't see the world the same way I do, as more than one person suggested that the spec was just fine and that "/" was not worthy of special consideration. We tried all sorts of common sense arguments, to no avail. In the end, we had to beat them at their own game, by pointing out that if one attempts to remove "/" recursively, one will ultimately attempt to remove ".." and ".", and that all we are doing is allowing rm to pre-determine this heuristically. Amazingly, they bought that!

Anyway, in the end, we got the spec modified, and Solaris 10 has (since build 36) a version of /usr/bin/rm (/bin is a sym-link to /usr/bin on Solaris) and /usr/xpg4/bin/rm which behaves thus:

[28] /bin/rm -rf /
rm of / is not allowed
[29] 

Oct 01 2004, 09:59:01 PM PDT Permalink Comments [9]

20040930 Thursday September 30, 2004
sendmail's new paradigm

Well, it's not all that new, as sendmail 8.12 was released over three years ago, but since few people pay attention to sendmail more than they have to, the major changes in 8.12 and their side effects may have gone unnoticed.

First, some background: sendmail has some historical notoriety from a security point of view, since it for many years was a setuid-root application, and thus a buffer overflow for example could make sendmail a vector for an unprivileged user to take over the system. Starting with version 8.12, however, sendmail lost its setuid bit in favor of becoming setgid to a new group.

To make queueing work, a split-queue model was created: /var/spool/mqueue as the traditional "server" queue with the same 750 root, bin permissions, and the new /var/spool/clientmqueue as the new (you guessed it) "client" queue with 770 smmsp, smmsp permissions. For the curious, "smmsp" is the new user and group invented for these purposes, having UID and GID values of 25 (chosen to match the SMTP port value in /etc/services), and it stands for "SendMail Message Submission Program".

The default configuration involving all "direct submission" messages [meaning messages submitted via sendmail invoked from the command line or comparably by programs such as mailx(1)] will go to the client queue. Messages in the client queue are then transferred to the SMTP server on the local host, which then proceeds in the traditional way. Thus running a sendmail daemon has become critical even for out-bound mail, at least from certain sources [such as cron(1M)]. See entry 3.44 of the sendmail FAQ for more details on this, as well as how to configure around this for hosts where accepting in-bound from other hosts is undesirable.


Sep 30 2004, 12:51:45 PM PDT Permalink Comments [1]

20040917 Friday September 17, 2004
Why HTML is bad for mail

Many people with whom I regularly correspond know that I have a strong preference against mail in "pure" HTML format. Here are my reasons, along with recommended alternatives.
  1. Spammers can hide tiny images: just a single pixel, small enough that your eye would likely miss it anyway, and the more insidious ones are the same color as the background. The URLs provided for your mail reader to fetch these images can contain encoded data which spammers can use to confirm your identity. I.e., each message they send can contain a slightly different URL, and when a mail reader fetches an image, it serves as confirmation for the spammers that the address corresponding to that particular URL is "live". This will in turn increase the likeliness of getting even more spam.
  2. Not all mail readers handle HTML well. Most modern mail readers do, especially those with a GUI, but many older mail readers, especially those which are screen-based, handle HTML badly or not at all. Some people may use a GUI mail reader at work but a screen-based one at home or when traveling.
  3. HTML takes up more bandwidth than plain text. Although this is not an issue in many environments (such as a high-speed LAN), in other environments (such a when traveling and stuck with a low-speed dial-up line) the extra bandwidth can be quite inconvenient.
The ideal alternative, when possible, is simply to send plain text, as it is sufficient for the vast majority of e-mail conversations.

When richer mark-up is needed, however, most mail programs which can generate HTML can also generate mixed text and HTML: the message's primary MIME type is multipart/alternative, with the first part being text/plain and the second part being text/html. Conforming mail readers will display HTML if they understand it, or plain text if they don't grok HTML (unless the user has configured it to display plain text by default).

For the curious, I use exmh at work and home but nmh (the CLI-based mail reading set of programs on which exmh is based) when on the road or in any other low-bandwidth environment. Exmh can display HTML, but it is much slower than displaying plain text, and not all constructs are well supported. I have mine configured to display plain text by default for multipart/alternative messages. I also have SpamAssassin configured to score "pure" HTML mail very highly, as the vast majority of such mail which I receive is indeed spam.


Sep 17 2004, 08:02:05 PM PDT Permalink

20040916 Thursday September 16, 2004
getting started
I post to Usenet occasionally about Zones, more often about sendmail. More to follow...

Sep 16 2004, 08:47:18 PM PDT Permalink