Wednesday May 07, 2008

uperf - opensource network benchmark
I noted that the availability of uPerf, has just been announced over on perf-discuss. I had the opportunity to play with uperf a little bit in my previous role, its an extremely powerful and useful network benchmarking tool. If you need to measure networking performance with "real world" type workloads (and lets face it thats what you need to be doing) uPerf is well worth checking out.
(2008-05-07 08:49:18.0) Permalink

Friday January 18, 2008

17th Irish OpenSolaris Users Group Meeting
Tim have been busy planning the next IE-OSUG meeting

Topic           News from December/January, Lightning talks
		& generally being social!

Date            Thursday January 31st
Time            7:00pm onwards
Location        DIT, Kevin Street, room KE: 1-008
	        (unless I hear otherwise from the nice folks at DIT)

This month, we don't actually have a main speaker organised, so we're
just going to wing it.


Along with presenting the OpenSolaris news which we didn't get to talk
about in December[1], we'll also cover January's news items, which are
already starting to mount up!

In addition, we could use the extra time to host some ad-hoc Lightning
Talks[2] - these worked pretty well the last time we did these, and we
think the casual atmosphere was a good one:
http://www.opensolaris.org/os/project/ie-osug/meetings/11/
If you've slides in advance that you want to send on, drop Tim a mail,
or feel free to just bring them along on the night.

We'll update this month's web page at:

http://www.opensolaris.org/os/project/ie-osug/meetings/17/

with any Lightning Talk topics we get in advance -- all volunteers
welcome!

As always, we'll be recording the talk and adding it to our
podcast feed as well as posting the slides here, in case you can't make
it to the meeting.

Note this month, we're holding the meeting on a Thursday this month -
we've had a few people say that Tuesdays don't suit, so perhaps it would
be better switching the day the meetings on each month to better
accommodate people ?

Many thanks to the nice folks at DIT for hosting our meeting, please let
me know if the change of date doesn't suit?


A link to this announcement is: 
http://www.opensolaris.org/os/project/ie-osug/meetings/17/
there's a PDF poster on the page you can use to further publicise the
meeting.

Friday December 21, 2007

DTrace ip Network Provider I see that Brendan Gregg has posted a draft PSARC case for the first part of the DTrace Network Provider over on DTrace-discuss. I can't even begin to describe how useful this will be.

Wednesday December 05, 2007

DTrace and the case of the slow login...

An interesting little problem came my way earlier on today which yet again let me show just how useful DTrace is. Way back when I did a bit of work with a customer around a small webapp they migrated to Solaris from Linux. Anyway I got a mail from them earlier today, they split out the webserver onto a seperate box a while back, and recently the initial connections to the database tier have gotten very slow, for no obvious reasons.

They were looking into things like large logs and slow writes etc, but nothing obvious was popping up. So we went through a couple of initial steps to see if anything obvious jumped out - the various *stat tools showed everything running normally so time to reach for DTrace. A quick oneliner showed up exactly where the problem was, and while its a simple fix I thought I'd share it as an example here. The key point here is that this isn't a resource starvation issue, which is when people generally reach for DTrace, rather its a nasty side effect of a simple change that doesn't manifest itself in an obvious manner. Right, enough typing, the oneliner was...

dtrace -n 'syscall:::entry { @sc[ustack()] = count(); } tick-5sec { printa(@sc); }'

Looking at the output the first thing that I spotted were some stacks similar to

			  libc.so.1`_so_send+0x7
              libresolv.so.2`send_dg+0xe5
              libresolv.so.2`res_nsend+0x45b
              libresolv.so.2`res_nquery+0xc9
              libresolv.so.2`ho_byaddr+0x379
              libresolv.so.2`ho_byaddr+0x80
              libresolv.so.2`gethostbyaddr_p+0x8e
              libresolv.so.2`res_gethostbyaddr+0x26
              nss_dns.so.1`_gethostbyaddr+0x21
              nss_dns.so.1`__nss_dns_getbyaddr+0x91
              nss_dns.so.1`getbyaddr+0x1b
              libc.so.1`nss_search+0x17d
              libnsl.so.1`_switch_gethostbyaddr_r+0x71
              libnsl.so.1`_uncached_gethostbyaddr_r+0x2a
              nscd`gethost_lookup+0x3dc
              nscd`0x80541da

              libc.so.1`stat64+0x7
              nss_files.so.1`getbygid+0x2e
              libc.so.1`nss_search+0x17d
              libc.so.1`_uncached_getgrgid_r+0x63
              nscd`getgr_lookup+0x3b2
              nscd`0x8054852
              libc.so.1`_thr_setup+0x4e
              libc.so.1`_lwp_start

Without DTrace the root cause would have been a lot more painful to find, instead this exchange was a couple of e-mails in the background while I got on with the day job. Gotta love DTrace ;).

Tuesday November 20, 2007

IPS @ 16th IE-OSUG
Tim has been busy organising the next Irish OpenSolaris Users Group meeting. The announcement mail is here, but for your ease of viewing..

I'm happy to announce the 16th Irish OpenSolaris User Group meeting:

Topic           Image Packaging System (IPS)
Date            Tuesday November 27th
Time            7:00pm onwards
Location        DIT, Kevin Street, room KE: 1-008

This month, Michal Pryc has kindly offered to give a talk on the Image
Packaging System[1] - a new package management system for OpenSolaris
that will ease the pain of package installation and upgrade.

The project is still in it's early stages, but you can try it out now by
downloading and experimenting with Indiana[2]. Come along to the meeting
to find out more!

Michal plans to give a quick outline of:

  * IPS structure
  * client/server side tools
  * limitations (some would say benefits ;-)
  * portability

as well as a GUI tool he's been working on. He's asked if people have
specific questions about IPS before the meeting, to feel free to post
them to the list and he'll try to prepare them for the meeting.

This should be a pretty interesting talk, and I'd definitely encourage
you to come along – please feel free to pass this message around and
spread the word.


As always, we'll also give a quick run down of OpenSolaris news
over the past month, and will be recording the talk and adding it to our
podcast feed as well as posting the slides here, in case you can't make
it to the meeting.

As always, many thanks to the nice folks at DIT for offering to host
our meeting!

A link to this announcement is:
http://www.opensolaris.org/os/project/ie-osug/meetings/16/


Look forward to seeing you there!   There's a poster on my blog if you'd
like to help us publicise this meeting.

        cheers,
                        tim

[1] http://opensolaris.org/os/project/pkg
[2] http://opensolaris.org/os/project/indiana/resources/getit

Wednesday October 17, 2007

Solaris Developer Express on Acer 5613 AWLMi
This post is a bit out of date as there have been several more builds of Solaris Express since I originally started to create this entry, but it should provide a good starting point for anyone with a similar laptop. While installing Solaris Developer Express on an Acer 5613 I ran into a couple of issues that took a bit of time to resolve, so rather than having someone else having to do the same digging I've listed the relevant details here. The installation discussed here was done with a Solaris Express Developer Edition 9/07 dvd.

Initial Boot

-B acpi-user-options=8

Networking

Wireless networking proved to be problematic, initially I had suspected something odd with the chipset, but it is supported. I logged 6614097 which after some analysis with the help of the wifi nic folks was shown to be a side effect of to the ACPI problems noted above.

This problem has been root caused to an inability to switch on the RF switch from Solaris due to having to disable ACPI, unfortunately there is no workaround for this. My solution has been to purchase an atheros based pcmcia wireless card which works perfectly.

I/O Resource Warnings

	WARNING: out of I/O resources on bridge: bus 0x20, dev 0x3, func 0x0, for secondary bus 0x23

Grub Entries for Windows & ACPI Workaround

# Unknown partition of type 6 found on /dev/rdsk/c0d0p0 partition: 2
# It maps to the GRUB device: (hd0,1) .
title Windows Vista
rootnoverify (hd0,1)
chainloader +1

#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title Solaris Express Developer Edition 9/07 snv_70b X86
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B acpi-user-options=8
module$ /platform/i86pc/$ISADIR/boot_archive
#---------------------END BOOTADM--------------------
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title Solaris failsafe
kernel /boot/platform/i86pc/kernel/unix -s -B acpi-user-options=8
module /boot/x86.miniroot-safe
#---------------------END BOOTADM--------------------

Thursday September 20, 2007

Ldoms @ IE-OSUG
The fourteenth Irish OpenSolaris Users Group meeting is on next Tuesday. Liam Merwick will be presenting on Logical Domains. If you have any machines with Coolthreads technology, this will definately be of interest to you.

The good folks at the Dublin Institute of Technology are hosting the meeting.

Logical Domains @ IE-OSUG
DIT, Kevin St, @ 7.00 PM, September 25th 2007

Thursday September 13, 2007

Stream and the performance impact of compiler optimization

Introduction

Now in order to keep the test in the realms of reproducability for most people (and of course within the bounds of not annoying other people by taking up valuable test cycles on some of our larger machines, I would have liked to give one of our Sun Fire X4600's a run with all of these experiments, but they are incredibly highly utilised by the various engineering groups we work with) the numbers which I based this post on were generated on a Sun Fire X2100 M2 Server, which is a single cpu, dual core amd64 box. The ARRAY_SIZE for our problems was set using the l2 cache size script mentioned previously.

Secondly, as mentioned before, my group does not generate benchmark numbers for publication, in general we run stream with -xopenmp -fast as a more out of the box type compilation, however here we are taking a baseline with no compiler options passed in, which consists of twenty iterations of stream, and calling that our 100% point, and finally expressing our results as a percentage of our baseline. Now with all of that aside, lets move onto the experiments.

Experiment Details

The compiler options used here do not represent an exhaustive comparison of the various compiler options, rather a more general indications of the kind of optimization that Studio 12 can do, and the impact that your compiler can have on the performance of your application. It should be noted though that Stream is a benchmark which is highly suited to be being optimized. For a more accurate set of results we use OpenMP. We have two tables of results below, one with OMP_NUM_THREADS set to the core count (two in this case) and one with OMP_NUM_THREADS set to the physical processor count (one in this case).

The Results

Compiler Options

Further Reading

Tuesday August 21, 2007

Quick grab of L2-Cache sizes on x64 for stream
One of the benchmarks we run on a regular basis is stream, which needs you to set up some feasible values based on the size of your L2 cache. Following a recent discussion over on perf-discuss I figured a bit of background and sharing was called for. Now we get access to a lot of random hardware, both new and old, particularly in the x64 space, and you generally just want to get things up and running without thinking about every small config detail, so enter smbios(1M). As an example the L2-Cache entry on a box I'm looking at currently is

ID    SIZE TYPE
6     27   SMB_TYPE_CACHE (processor cache)

  Location Tag: L2-Cache

  Level: 2
  Maximum Installed Size: 1048576 bytes
  Installed Size: 1048576 bytes
  Speed: Unknown
  Supported SRAM Types: 0x10
        SMB_CAT_PBURST (pipeline burst)
  Current SRAM Type: 0x10 (pipeline burst)
  Error Correction Type: 5 (single-bit ECC)
  Logical Cache Type: 5 (unified)
  Associativity: 5 (4-way set associative)
  Mode: 2 (varies by address)
  Location: 0 (internal)
  Flags: 0x1
        SMB_CAF_ENABLED (enabled at boot time)

Now smbios is incredibly useful, but it still doesn't give us a generic solution that works most of the time for the task at hand, so enter Perl, and getl2cachesize. Its short and sweet, but if your running stream it will give you a good starting point for your array sizes. A couple of other pointers, I'll leave compiler optimizations for your on reading, but as a warning stream is very variant, so get a lot of runs, generally we get twenty iterations, average that as a single run, reboot, and get at least five runs.
(2007-08-21 03:39:07.0) Permalink

Tuesday August 14, 2007

DTrace for #!/bin/sh
Now this is seriously handy - Alan Hargreaves has released a DTrace provider for sh over on opensolaris.org. No more hiding under the excuse "its the shell" - bad shell scripts will now be exposed :).
(2007-08-14 09:35:31.0) Permalink

Thursday August 02, 2007

13th Irish OpenSolaris User Group Meeting
Tim has been busy organising the next Irish OpenSolaris users group meeting -

To: [ug-ieosug] 13th Irish OpenSolaris User Group Meeting
From: Tim Foster Tim.Foster at Sun.COM
Date: Wed Aug 1 08:42:29 PDT 2007

Hi all,

I'm happy to announce the 13th Irish OpenSolaris User Group meeting:

Topic		Discovering Open High Availability Cluster
Date		Tuesday	August 14th
Time		7:00pm onwards
Location	TBD

This month, we're lucky to have Nick Solter, from the Open HA Cluster
community visiting Dublin who will give us a talk about Open HA Cluster,
which forms the basis of one of the newer communities on
opensolaris.org.

Solaris Cluster, Sun's high-availability cluster software, is going
open-source! This presentation will first give an overview of High
Availability software and Solaris Cluster, and demonstrate how Open HA
Cluster can be used with OpenSolaris and the Apache webserver to provide
higher availability for web services.

Nick's talk will next describe the open-source roadmap, give an overview
of the source code, and walk through the steps to download, compile,
install, and play with Open HA Cluster and Cluster Express. The talk
will conclude with a description of the contribution process and some
sample projects for community participation.


As usual we'll also be doing the roundup of news since our last talk,
and will be adding audio from this meeting to our podcast feed.

More info at:
http://www.opensolaris.org/os/project/ie-osug


We *don't* yet have a venue for this meeting, so if you'd like to host
it, please drop a mail to this list [ I'm actually out on vacation from
Monday for 2 weeks, but will of course be at the meeting on the 14th and
will try to monitor this list in the meantime: so please could someone
finalise the venue for the group - otherwise, I'll assume we're at
meeting at the Sun offices in East Point as per last month. ]


I'd also like to take this opportunity to poll for talks or speakers for
our September meeting - if you'd like to talk about anything
OpenSolaris-related on Sep 25th, we want to hear from you.

If you're looking for an excuse to visit Ireland, here's your chance -
we'd love to have you present a talk to the user group, and will do our
best to show some Irish hospitality!

See you all then!

Thursday July 12, 2007

Project Indiana @ IE-OSUG Tonight
Just a quick reminder that Glynn Foster is talking about Project Indiana, the project to create a reference OpenSolaris distro, at tonights Irish OpenSolaris Users Group Meeting.

Tonights meeting is being held at the Sun Ireland office in Eastpoint Business Park - just a short walk from Clontarf Road Dart Station. Tim has already created map.

Thursday July 05, 2007

12th Irish OpenSolaris Users Group Meeting - Project Indiana
Tim's been busy sorting out speakers again for the next Irish OpenSolaris Users Group Meeting ...

Subject: [advocacy-discuss] 12th IE-OSUG meeting - Project Indiana
From: Tim Foster 

Hi all,

We're happy to announce the 12th Irish OpenSolaris User Group meeting.

    Topic: Project Indiana
     Date: Thursday 12th July 2007
     Time: 7pm onwards
 Location: TBD

July's meeting is a little earlier than usual: we're delighted that
Glynn Foster will be in the country and has agreed to give a talk on
Project Indiana, a project that aims to create a new distribution of
OpenSolaris that's more familiar to users of GNU/Linux.


If you've been keeping up with OpenSolaris, you'll know that this has
been a highly anticipated, and much talked about project, and it's an
exciting time for OpenSolaris at the moment.

Along with his role in helping to define the Indiana project, Glynn was
elected onto the OpenSolaris Governance Board (OGB) during the elections
in May this year, so you should use this opportunity to pick his brains
while he's in this hemisphere!

- more about Project Indiana, and the latest draft of the Problem
Statement at:
http://www.opensolaris.org/os/project/indiana
http://www.opensolaris.org/os/project/indiana/documents/problem_statement/


This announcement is available at:
http://www.opensolaris.org/os/community/os_user_groups/ie-osug/iosug-12/
please feel free to spread this announcement as widely as you can - this
is a rare opportunity to hear from someone on the OGB.


As usual, we'll be recording this meeting, and adding it to our podcast,
which is available at:
http://mediacast.sun.com/share/timf/ie-osug-podcast.rss


We haven't yet finalised a venue for this meeting - if we have to, we'll
hold it in the Sun offices in East Point Business Park near Clontarf,
but if anyone else would like to offer a venue that's a bit more
central, please let us know!  I'll send an update on Tuesday next week.

		cheers,
				tim

Tuesday June 26, 2007

ZFS Test
Jim Walker has announced the arrival of the zfs test suite over on testing-discuss. Mike Byrne has a brief write up on the cast.
(2007-06-26 06:04:13.0) Permalink

Five DTrace Oneliners in Five Minutes
The 11th Irish Opensolaris Users Group meeting is taking place tonight, and I have offered to give one of the lightning talks. So, without further ado, the slides for my lightning presentation this evening

Five DTrace Oneliners in Five Minutes

Now if you have clicked on the slides you will have noticed that its not quite five oneliners, but seeing as the DTrace consumers I'm using fit on one line I think they qualify :).