As Good A Place As Any

Who SED that ?

I recently had to quickly write a filter to convert a pile of test data into CSV format for upload into a spreadsheet, and I found this excellent reference: Handy One-Liners For SED. This was real help and I recommend it for people in a  hurry, as I was.

But, it made me think: I do own books on SED and AWK...maybe it is time to read them again...

Posted at 07:08PM May 17, 2008 in Miscellaneous  |  Comments[0]

root user seeks new home....

I was very confused recently after installing snv b87.

I had used my usual JET template to configure the jumpstart installation, but on logging in none of my commands worked...

So, what's up with that ? Did jumpstart screw up ? Well, actually not.

It turns out that as of build 87 root's home directory is /root not / and root's .profile file etc needs to be in /root.

Well..that is something that I thought would never change! However, I can imagine the number of occurrences of people accidentally deleting parts of the operating system will fall dramatically .... not that I would ever do anything that dumb...well it only happened once...honest

Posted at 08:52AM Apr 25, 2008 in OpenSolaris  |  Comments[0]

On Losing My Father.....

It is rare that I make a personal entry in this blog, but this lunchtime I took a walk through the Botanical Gardens close to my home and I was moved to write something about the recent death of my father, who had shared a love of nature and gardening with my mother.

The relationship with one's parents can be difficult and since his death I have tried not to remake history, but I have often thought what would I say about my father if asked and here it is...

Dennis Victor Thomas: Born Feb 1st 1922, Died February 24th 2008.

At various times in his life he was a soldier, a lorry driver, a farmer, a welder, a carpenter and a builder...and for most of his life he was a father.

A veteran of World War II, he was a driver in the 8th Army through North Africa and Italy. He was present at the Battle of Monte Casino, something he only mentioned once, as is the way of his generation.

He was a father to four sons: Paul, Christopher, Michael and Timothy (me).

My parents were intensely private people so I shall share no more of their lives other than to say that he died almost four years to the day after my mother, his wife of 57 years.

Of my father: I shall just say that he was a quiet and gentle man with a good kind heart...and he did his best. Uncomfortable with his emotions, he was a silent provider for his family. I only found out of his pride in me from others. He showed his love for his grandson, our 8 year old, by saving his meager pension and quietly delighting in giving him the money when we visited him.

A lovely poem was read at the end of my mother's funeral service. I was surprised to find a cutting of it in my fathers wallet after he died, and it was read at the end of his funeral also. It is a famous inspirational poem attributed to Mary Frye.

Do Not Stand At My Grave And Weep

Do not stand at my grave and weep
I am not there; I do not sleep.
I am a thousand winds that blow,
I am the diamond glints on snow,
I am the sun on ripened grain,
I am the gentle autumn rain.
When you awaken in the morning's hush
I am the swift uplifting rush
Of quiet birds in circling flight.
I am the soft starlight at night.
Do not stand at my grave and cry,
I am not there; I did not die.

[I have disabled comments on this entry.]

Posted at 03:00PM Apr 18, 2008 in Personal  | 

OpenSolaris as a StorageOS - The Week That Everything Worked First Time

This has been an extraordinary week...everything I have tried to do has worked first time.

To summarise this weeks activities, I have set-up and then blogged on:

Configuring the OpenSolaris CIFS Server in Workgroup Mode
Configuring the OpenSolaris CIFS Server in Domain Mode
Solaris CIFS Windows SID Mapping - A First Look
Configuring the OpenSolaris Virus Scanning Services for ZFS Accessed via CIFS Clients

All in all..a very good week  

Posted at 09:48AM Apr 11, 2008 in OpenSolaris  |  Comments[0]

Configuring the OpenSolaris Virus Scanning Services for ZFS Accessed via CIFS and NFSv4 Clients

OpenSolaris includes features that enable virus scanning of files accessed by CIFS and NFSv4 clients. You can read about the project on the OpenSolaris Project: VSCAN Service Home Page. The scanning model is similar to that discussed in this article in that files are sent (using the ICAP protocol) to external servers running anti virus software to be scanned...a common model for NAS appliances.

Having set up the OpenSolaris CIFS server as described here, I wanted to try out these services. Part of my job at Sun is to certify anti virus software with our NAS products, so I am experienced in this kind of testing.

As before, I am working on a Sun Fire X4500 with Solaris Nevada build 86 installed....

root@isv-x4500b # uname -a
SunOS isv-x4500b 5.11 snv_86 i86pc i386 i86pc

I have mostly presented the commands I have used as is..but I have occasionally had to edit fields.

I installed a copy of the Symantec Scan Engine Version 5.1 on a Windows server (hostname: scanengine) in my lab to provide the necessary scanning services. The Symantec Scan Engine has an ICAP protocol interface as standard, so enabling the OpenSolaris VSCAN service to communicate with it.

I configured the Symantec Scan Engine in "Scan Only" mode which means that it will notify the VSCAN service if a file is infected or not..it will not attempt to repair infected files (i.e. if won't remove the virus). The response of the VSCAN service to being told that a file is infected is to set attributes on the file so that access is denied i.e. the file is quarantined.

The VSCAN services are managed with the vscanadm command. The vscan service daemon, vscand, interacts with the scan engine to have the file scanned; sending the file contents to the scan engine via the ICAP protocol.

I wanted to enable virus scanning on a CIFS share called cifs2 which is a share off the ZFS file system tank/cifs2. The procedure was as follows:

1. Enable VSCAN Services

root@isv-x4500b # svcadm enable vscan
root@isv-x4500b # svcs vscan
STATE          STIME    FMRI
online          7:38:08 svc:/system/filesystem/vscan:icap

2. Enable Virus Scanning on the ZFS File System

root@isv-x4500b # zfs set vscan=on tank/cifs2

NOTE: In the next steps you should substitute the hostname of your server running anti virus software for scanengine.

3. Add a Scan Engine

root@isv-x4500b # vscanadm add-engine scanengine
root@isv-x4500b # vscanadm get-engine scanengine
scanengine:enable=on
scanengine:host=scanengine
scanengine:port=1344
scanengine:max-connection=32

NOTE: Port 1344 is the default for ICAP, it can be changed. If you changed it you would also need to change the port used by the anti virus software.

4. Optional Step - Set Maximum Size of File To Sent To Be Scanned

It is inefficient to scan very large files. Here we set maximum size of a file to be scanned to 100 MB. If a file 100MB in size, or greater, needs to be scanned you have an option of allowing access or denying access: in either case the file will not be scanned.

root@isv-x4500b # vscanadm set -p max-size=100M
root@isv-x4500b # vscanadm set -p max-size-action=deny
root@isv-x4500b # vscanadm show
max-size=100M
max-size-action=deny
types=+*

scanengine:enable=on
scanengine:host=scanengine
scanengine:port=1344
scanengine:max-connection=32

5. Optional Step - Modify The List of Types of File to Scan

By default all file types are scanned. Here we remove files ending in "jpg" from the list of files types to be scanned.

root@isv-x4500b # vscanadm set -p types=-jpg,+*
root@isv-x4500b # vscanadm show
max-size=10M
max-size-action=deny
types=-jpg,+*

scanengine:enable=on
scanengine:host=scanengine
scanengine:port=1344
scanengine:max-connection=32

6. Checking That Scanning is Working

You can get files from EICAR to test virus scanners. The files look like viruses to the scanners, but are safe to use.

I mounted up the cifs2 share on a Windows server and created some files and Folders on the share with no issues. I then drag and dropped EICAR files from the Window server's unprotected drive onto the share. When I tried to open an EICAR file on the share, access was denied.

Messages like the one below appeared in the system log.

Apr  9 08:13:09 isv-x4500b vscand: [ID 540744 daemon.warning] quarantine /tank/cifs2/eicar.com.txt 11101 - EICAR Test String

Back on the server running OpenSolaris I checked to see if files had actually been scanned as below

root@isv-x4500b # vscanadm stats
scanned=13
infected=6
failed=0
scanengine:errors=0

File are being scanned and there are no scan errors! 

Lastly, I also looked at the statistics on the Symantec Scan Engine GUI which also confirmed that files were being scanned. Note that the numbers below do not match the output of vscanadm above as the screenshot was taken a a later time.

Access is denied to the infected files because the quarantine bit has been set. You can check for the q for quarantine bit as follows...

root@isv-x4500b # ls -/c eicar.com.txt
----------+  1 2147483649 2147483650      68 Apr  9 08:13 eicar.com.txt
                {A------mq-}

For More Information

OpenSolaris Project: VSCAN Service Home Page

vscanadm and  vscand manual pages.

Posted at 09:01AM Apr 11, 2008 in OpenSolaris  |  Comments[1]

Solaris CIFS Windows SID Mapping - A First Look

This follows directly on from where I set up an OpenSolaris CIFS server in Domain Mode. When I mapped the share I was not asked for a user ID and password as I had been in Workgroup mode..so, how are file and folder ownership and permissions handled ?

The Solaris CIFS implementation includes ID Mapping Services. These services let you explicitly map Windows Security IDentifiers to Solaris User IDs and Group IDs if you wish, but if you don't set up explicit mappings then the ID Mapping Service will generate Ephemeral Solaris User IDs and Group IDs for Windows Users. The Solaris CIFS Administrators Guide discusses Identity Mapping strategies.

Permissions on the files and directories created at the end of the previous exersize, when listed from Solaris, look like this...

root@isv-x4500b # pwd
/tank/cifs1
root@isv-x4500b # ls -l
total 10
d---------+  2 2147483649 2147483650       2 Apr  8 05:49 user1
----------+  1 2147483649 2147483650       0 Apr  8 05:57 user1.txt
d---------+  2 2147483650 2147483650       3 Apr  8 05:50 user2
----------+  1 2147483650 2147483650       0 Apr  8 05:51 user2.txt

This looks a bit odd because the OpenSolaris CIFS service uses ZFS ACLs, and they don't show up in the above listing. We can dump the current ID Mappings, as below...

root@isv-x4500b # idmap dump
usid:S-1-5-21-500772251-2770406677-2360070262-1125      ==      uid:2147483650
usid:S-1-5-21-500772251-2770406677-2360070262-1113      ==      uid:2147483649
gsid:S-1-5-21-500772251-2770406677-2360070262-513       ==      gid:2147483650
gsid:S-1-5-21-500772251-2770406677-2360070262-512       ==      gid:2147483651
gsid:S-1-5-11   ==      gid:2147483652
gsid:S-1-5-2    ==      gid:2147483653
gsid:S-1-5-32-544   ==      gid:2147483654

We can see that Windows SIDs have been mapped to Solaris User and Group IDs.  The mappings for the users have been created on the fly (Ephemeral ID Mapping); the Windows Group SID mappings are generated by system level interactions, you can decode those using the list here. We can see that:

Windows Group SID xxxxxx-513 = Domain Users = Solaris GID 2147483650

Windows Group SID xxxxxx-512 = Domain Admins = Solaris GID 2147483651

Windows Group SID xxxxxx-11 = Authenticated Users = Solaris GID 2147483653

Windows Group SID xxxxxx-2 = Network (Users logged in via the Network) = Solaris GID 2147483651

Windows Group SID xxxxxx-544= Administrators  = Solaris GID 2147483654

We now know that the files and directories we created are in the Domain Users group as it was mapped to the Solaris GID of 2147483650.

Sometime soon I am going to dig deeper on all this, this was just a first look. 

For More Information

OpenSolaris Project: CIFS Server Home Page

Open Solaris CIFS Documentation including the Solaris CIFS Administrators Guide & Troubleshooting Information

Also, consider joining the Open Solaris Storage Discuss Forum

Posted at 09:20AM Apr 10, 2008 in OpenSolaris  |  Comments[0]