If I could just get the stupid thing to build

Old and new friends at LOSUG

Went to London for the LOSUG meetup last night. Had a really fantastic time and met lots of great people. Thanks to James and Joy for organising my session and for inviting me along to talk. Met up with Ralph Turner from OpenAnswers who I've not seen for an age, Ralph used to do work for us when I was in Developer Tech Support and we see each other now and again, but not often enough. My former boss Dani Flexer was there and he seems to be doing really well for himself, he was a speaker at a previous LOSUG meeting in September. I have to read his white paper when I get the chance. I also met Robert Milkowski who is a regular contributor to SourceJuicer (he recently contributed Alpine). We had never met before so it was great to put a face to the name. I don't remember the names of everyone that I met last night but it was really, really good to get the chance to meet you all.

Owen Roberts' gave a great talk on creating an OpenSolaris build server from IPS images, it gave me a sense of DejaVu as so much of it was the same as what I'm doing with updating Lighttpd and Ruby for WebStack in OpenSolaris.

The slides for my talk are here, but they are fairly minimalist (which works so much better for me as a presenter). I'll also record the demo that I gave and post that on slx.sun.com.  I enjoyed giving the presentation immensely and I hope that everyone got as much from it as I did. I really, really look forward to going back in the the new year and going into the technical details of publishing a package into /contrib.

Also, a shout out to the SourceJuicer team in Dublin and to Jim Walker in California. They had a network outage during my afternoon yesterday that meant that the SourceJuicer site was unavailable and the systems couldn't be accessed at all. Everyone worked together to get it back up and running so that I could run my demo, keeping me informed by phone and by email. Thanks for that guys :o)

Posted at 11:08AM Nov 19, 2009 by MandyWaite in Events  |  Comments[0]

SourceJuicer at London OpenSolaris User Group meetup

I'll be giving a talk on SourceJuicer at the London OpenSolaris User Group meeting on Wednesday 18th November. On before me is Owen Roberts from the Solaris Sustaining group, and he'll be talking about how the core of OpenSolaris will move to being built from source directly to IPS packages (yay!).

So if you are interested,  register in advance here and then come along to the Sun Customer Briefing Centre at:

45 King William Street
London, EC4R 9AN
Map

Doors open at 6PM and the talking starts at 6:30PM, there'll be food and socialising afterward.

While on the subject of SourceJuicer, George Drapeau the OSS core team lead here in ISV-Engineering at Sun has just published a video blog on installing Acquia Drupal on OpenSolaris from the /contrib repository. You can find it on George's Blog, it's very slick :o)

Posted at 01:56PM Nov 12, 2009 by MandyWaite in Events  |  Comments[0]

Time for a new Alfresco on Glassfish walkthrough

My Alfresco on Glassfish Walkthrough is over a year old now and is creaking at the seams a little so I plan to update it. Feel free to comment here on what the new walkthrough should cover, Glassfish v2.1 or v3? Solaris or OpenSolaris? Linux or Windows? Which DB (just MySQL or some other OSDB)?

In fact feel free to comment on anything you would like to see to do with running Alfresco (or any other (E)CMS app) on Sun gear.

Original blog entry: http://blogs.sun.com/mandy/entry/alfresco_on_glassfish_on_opensolaris

Cheers

Amanda

Posted at 11:39AM Nov 11, 2009 by MandyWaite in Open Source  |  Comments[2]

autoconf weirdness

configure.ac files (configure.in in the old days) are used as input to autoconf which miraculously (it's all miraculous to me) expands the macros in the configure.ac file and creates a runnable configure shell script. Autoconf macros may be system dependent so if you decide to regenerate a configure script, it's the job of autoconf, in conjunction with aclocal, to locate all of the macro definitions that it's going to need to get the job done. These macro definitions may be supplied by the local installation of autoconf, by other packages via aclocal or actually supplied by the package you are building in a file named acinclude.m4

Due to some changes in the way Lighttpd is built I found that for Lighttpd 1.4.23 in WebStack we had to regenerate the configure script, I did this using the autogen.sh script that ships with the lighttpd source code. autogen.sh calls libtool, aclocal, automake, autoconf and autoheader. On Solaris 10 autoconf generated an error:

configure.ac:67: error: possibly undefined macro: AC_DEFINE
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.

But AC_DEFINE is a standard autoconf macro so why can't it be found? We dug around and found mention of it being to do with pkg-config but couldn't work out why that would be. So We stripped down the lighttpd configure.ac bit by bit and found that the problem went away when we removed the lines referencing a macro named PKG_CHECK_MODULES. The file aclocal.m4 in the lighttpd build directory contains any additional macros required by autoconf to process configure.ac and is generated by aclocal. The lighttpd source distribution includes the aclocal.m4 file that was used to create the original configure script, but when you run aclocal, a new aclocal is generated from macros that have been registered with aclocal on the local system. On this system, after running aclocal, aclocal.m4 didn't contain a macro named PKG_CHECK_MODULES :o/

Further research showed that PKG_CHECK_MODULES is usually supplied in a file named pkg.m4 which is installed as part of pkg-config. It usually installs to /usr/share/aclocal which is the default location used by aclocal to locate macros. In this case though we were using a tool set specifically created by the WebStack team for building WebStack on Solaris 10. In this tool set (wstools),  aclocal lives in /export/wstools/bin and it looks for macro definitions in /export/wstools/share/aclocal. wstools doesn't include pkg-config so there's no pkg.m4. That explains why it couldn't find the PKG_CHECK_MODULES macro, but why the error about AC_DEFINE?

In configure.ac you'll find:

  if test "x$FAM_LIBS" = x; then
    PKG_CHECK_MODULES(FAM, gamin >= 0.1.0, [
      AC_DEFINE([HAVE_LIBFAM], [1], [libfam])
      AC_DEFINE([HAVE_FAM_H], [1], [fam.h])
    ])
  fi

So you can kind of see why the AC_DEFINEs passed to the PKG_CHECK_MODULES macro could be affected, but autoconf indicated that the error came from a line of code not related to PKG_CHECK_MODULES, i.e.:

AC_CHECK_MEMBER(struct tm.tm_gmtoff,[AC_DEFINE([HAVE_STRUCT_TM_GMTOFF],[1],[gmtoff
in struct tm])],,[#include <time.h>])

Here AC_DEFINE is used with another standard autoconf macro (AC_CHECK_MEMBER) and if you look at the generated configure script you find:

if test "x$ac_cv_member_struct_tm_tm_gmtoff" = x""yes; then

cat >>confdefs.h <<\_ACEOF
#define HAVE_STRUCT_TM_GMTOFF 1
_ACEOF

fi

Which shows that it didn't actually fail where it said it did. Turns out the time it failed to process AC_DEFINE were where it was used with PKG_CHECK_MODULES. The error from autoconf actually referenced the first use of AC_DEFINE and not where it failed. Best of all, rather than emit a warning about PKG_CHECK_MODULES being missing, it just adds this to the configure script:

  if test "x$FAM_LIBS" = x; then
    PKG_CHECK_MODULES(FAM, gamin >= 0.1.0,
      AC_DEFINE([HAVE_LIBFAM], [1], [libfam])
      AC_DEFINE([HAVE_FAM_H], [1], [fam.h])
    )
  fi

Which is the un-expanded macro calls from configure.ac. These aren't valid shell commands, so the generated configure script will fail. In this case the best way to fix the problem is by adding pkg-config to the WebStack build tool set.  I've also heard that this problem can occur on systems with older versions of pkg-config, I guess that they may not define a PKG_CHECK_MODULES with the signature required by the configure.ac script.

Posted at 06:43PM Nov 05, 2009 by MandyWaite in Open Source  |  Comments[0]

Installing OpenSolaris packages from /contrib and /pending

The OpenSolaris /contrib package repository has a whole bunch of packages contributed by members of the OpenSolaris community or just by those who had a particular need for a certain package and who decided to submit it themselves.

You can install any of these packages either using the pkg command or via the package manager, you can even just go to the /contrib repository's catalog page and click on the 'install' link for the package(s) that you want to install.

To set up /contrib as a valid publisher for use with the package manager or the pkg command do the following as a user with the root role (or Software Installation profile):

pfexec pkg set-publisher -O http://pkg.opensolaris.org/contrib  contrib

Or you can add /contrib via the package manager through the File -> Manage Repositories pull down. If installing from the "install" link at http://pkg.opensolaris.org/contrib/en/catalog.shtml  the package manager will automatically add /contrib to the list of publishers.

To install packages using pkg, do the following:

pfexec pkg refresh

pfexec pkg install <package name>

You don't have to run 'pkg refresh' before installing every package, but it is probably best to do it reasonably regularly just to make sure that pkg has an up to date view of the repositories that it knows about.

There is also a /pending repository which is used to stage packages for testing while they are being made ready for promotion to /contrib. I wouldn't recommend arbitrarily using packages from /pending but you may want to provide feedback on a package, or as in the case with the ruby-mysql package, it maybe that the owner has suggested that you get it from /pending due to some issue getting it into /contrib.

To setup /pending as a publisher, you do much the same as you do for /contrib:

pfexec pkg set-publisher -O http://jucr.opensolaris.org/pending  pending

Or use the package manager File -> Manage Repositories menu.

To list the repositories that pkg knows about, run:

% pkg publisher 

PUBLISHER                             TYPE     STATUS   URI

opensolaris.org          (preferred)  origin   online   http://pkg.opensolaris.org/
contrib                               origin   online   http://pkg.opensolaris.org/contrib/
pending                               origin   online   http://jucr.opensolaris.org/pending/
mypkgs                                origin   online   http://localhost:80/
webstack                              origin   online   http://pkg.opensolaris.org/webstack/

If you have both /contrib and /pending added as publishers, when you install packages you'll need to qualify the package name with the name of the publisher from which you want to install it as follows:

pfexec pkg install pkg://pending/ruby-mysql

Which will install the 'ruby-mysql' package from the publisher name pending. Note that this is the publisher name, not the URI of the repository.

In the list of repositories above are a couple of other repositories that we've not talked about. The /webstack repository has packages that the WebStack team feel might be useful, but for which they are unable to offer support. This has packages for Web Tier applications such as Varnish and Nginx. We also sometime publish packages that are in the process of being integrated into OpenSolaris but are not yet available in the main repositories. The mypkgs repository is a repository local to our test system. It is very easy to set up a build environment for building your own packages from Spec files and then to publish them to a local repository. If you're going to do that though, you might as well publish them to /contrib via SourceJuicer.

Here's some other links that you might find useful:

Spec Files Extra is a project centered around the pkgbuild tool that builds Solaris SVR4 and/or IPS packages from spec files (and which is used by SourceJuicer)

The Genunix site has a page describing how to setup a SourceJuicer like environment

If you'd rather not get involved in submitting spec files to SourceJuicer but would like to see a package in OpenSolaris, drop us an email at sw-porters-discuss@opensolaris.org.

Posted at 11:26AM Oct 29, 2009 by MandyWaite in Open Source  |  Comments[0]

Ruby MySQL nearly in OpenSolaris

One of the big pain points when installing Native Ruby Gems is the need to have various build packages installed. Packages that deliver the likes of gcc, gmake and ginstall. You also need to know where the libraries and C header files that you want to build against are located. On OpenSolaris the last part should be a no-brainer at least with packages installed from the repository, but some packages such as MySQL don't install to /usr/lib and /usr/include and the mysql_config that the MySQL package ships is not on the default $PATH and even if it was, it emits Compiler and Linker information for Sun Studio, not for gcc.

What this means is that you have to install all of the tools above and then install the MySQL gem with options telling the build where to find the MySQL libs and the MySQL headers.

Making the MySQL gem available as an OpenSolaris package, means you don't have to worry about any of that. You just run:

pfexec pkg install ruby-mysql

and voila!, you have MySQL support in Ruby... But it doesn't work :o(

The ruby-mysql package was promoted to the /contrib repository this week. Unfortunately it was promoted before I had tested it fully (which shows we have some major holes in the processes used to get packages into /contrib). The version that's there currently is unusable as it causes a segmentation violation when running with Rails. If you want to use this package today then you can get it from the /pending repository. Details of how to make use of OpenSolaris repositories can be found on a separate blog entry here.

Posted at 10:49AM Oct 29, 2009 by MandyWaite in Ruby (on Rails)  |  Comments[0]

Nagios, Octave and SilverStripe in OpenSolaris

Several interesting packages were published to the OpenSolaris /contrib repository yesterday. They include Octave, SilverStripe and the 3 packages that make up Nagios. 

Nagios is a leading Open Source infrastructure monitoring tool that can monitor networks, hosts and even services, in fact it can monitor pretty much anything and it being Open Source it's fairly straightforward to add your own plugins. The main Nagios package is simply named nagios and delivers version 3.0.6 currently. In addition there is the nagios-plugins package which you'll install on systems that you want to monitor. A third package is nrpe (Nagios Remote Plugin Executor), which allows a centralised deployment of Nagios to execute and monitor plugins on other systems.

Octave is a GNU project, it's a high level language and runtime whose main use is for numerical computations. It's mainly command line driven, but can hook up with the likes of gnuplot in order to present graphs and other visual forms of mathematical data. I've tested it and it looks like it could be really useful for someone say, studying for a maths degree... like me for instance :o)

SilverStripe is a popular Open Source Content Management System, it's one of a number of such applications that OpenSolaris users have been clamouring for. Interestingly we did have some problems getting it to play nicely with MySQL 5.1 on OpenSolaris. When you are setting up the MySQL database prior to configuring SilverStripe modify the MySQL SMF service as follows:

svccfg -s mysql:version_51 setprop mysql/enable_64bit=true
svcadm refresh mysql:version_51
svcadm restart mysql:version_51

You'll either need to do this as root or as a user with the root role. These changes cause MySQL 5.1 to run in 64-bit mode.

You can install any of these packages either using the pkg command or via the package manager, you can even just go to the /contrib repository's catalog page and click on the 'install' link for the package(s) that you want to install.

To set up /contrib as a valid publisher for use with the package manager or the pkg command do the following as a user with the root role (or Software Installation profile):

pfexec pkg set-publisher -O http://pkg.opensolaris.org/contrib  contrib

Or you can add /contrib via the package manager through the File -> Manage Repositories pull down. If installing from the "install" link at http://pkg.opensolaris.org/contrib/en/catalog.shtml  the package manager will automatically add /contrib to the list of publishers.

To install packages using pkg, do the following:

pfexec pkg refresh

pfexec pkg install <package name>

These packages are great additions to the growing list of packages available via the /contrib repository. If you would like to contribute a package to /contrib visit the SourceJuicer page on OpenSolaris.org. If you'd rather not get involved in submitting spec files to SourceJuicer but would like to see a package in OpenSolaris, drop us an email at sw-porters-discuss@opensolaris.org.

Posted at 05:47PM Oct 28, 2009 by MandyWaite in Open Source  |  Comments[0]

Rake should be on the default PATH...

I can convince myself of anything. When we updated from RubyGems 0.9.4 to 1.3.5 I decided that it was ok for the 'rake' command to be in the $GEM_HOME/bin which then translated to /var/ruby/1.8/gem_home/bin. I justified it because users could run 'gem env', look at the "EXECUTABLE DIRECTORY" and see that they had to add /var/ruby/1.8/gem_home/bin to their $PATH env var. Having had to install and then run rake several times on different systems over the last few days I can see that this is wrong. I want to 'gem install rake' and then run 'rake <some rake task>' without the need to update my PATH. I'm certain that others will agree that this is how it should be.

I could easily change the way that we build RubyGems such that the "EXECUTABLE DIRECTORY" defaults to /usr/bin but I have no way of being certain that when the end user runs 'gem install <some gem with an executable>' that /usr/bin will actually be writable even by root (or the root role as used by pfexec). We have the Rails package in the /contrib repository that also delivers the Rake gem. This package installs the 'rake' executable to /usr/bin, but it's a bit much to ask you to install Rails just to get Rake. I seem to have the following options:

1. With the SUNWruby18 package on OpenSolaris, provide wrapper scripts in /usr/bin for the popular gems that deliver executables. These would emit a message saying to install the require Gem should it not already be installed and run as normal if it is. This has a few corner cases that have to be considered
2. Provide a /contrib package that provides wrapper scripts for popular gems which would work as described in (1)
3. Change "EXECUTABLE DIRECTORY" to /usr/bin and modify RubyGems to detect the writability of /usr/bin. It does this by default but would end up installing any affected gems in root's ~/.gem directory.

I favour option 3, but only if it can be done without significant changes to RubyGems.

Posted at 01:19PM Oct 14, 2009 by MandyWaite in Open Source  |  Comments[2]

Rails: Setting selected options in a ListBox

I really struggled with this yesterday. I'm writing a Rails CRUD app and when creating a new entry one of the fields presented is a ListBox that allows multiple selections. That was straightforward, but i had problems when editing an existing entry. What didn't want to work for me are the recognised methods for having the current items for the entry being edited be selected in the ListBox.

The way that I understand that it should work is like this:

collection_select(:user, :group_ids, Group.find(:all), :id, :name,{},{:multiple=>,:name=>'user[group_ids][]'})

But needless to say it didn't. In the end I found this from Wes Gamble (weyus on Ruby Forum). It's a bit more DIY but it works a treat. I'm reproducing it here in case it's useful to anyone else (thanks Wes!)

<select id="user_roles" name="roles[]" multiple="multiple">
<% Role.find(:all).each do |r| %>
     <option value="<%= r.id %>" <%= @user.roles.include?(r) ? 
"selected=\"selected\"" : '' %>"><%= r.name %></option>
<% end %>
</select>

Posted at 12:05PM Sep 25, 2009 by MandyWaite in Ruby (on Rails)  |  Comments[0]

Other packages in /contrib

Just following on from my last post, we've also added FreeImage to the OpenSolaris /contrib repository. This is pretty useful as FreeImage is used by the image_science ruby gem for manipulating images, particularly creating Thumbnails from uploaded images. FreeImage can be difficult to build on OpenSolaris and the 'freeimage' OpenSolaris package is fully built and includes both a static library and a shared library that can be used to build against when developing applications. You can install this package with the following commands (the first isn't needed if you've already added /contrib as a publisher) :

% pfexec pkg set-publisher -O http://pkg.opensolaris.org/contrib  contrib

% pfexec pkg refresh

% pfexec pkg install freeimage

If you then install the image_science gem (along with the RubyInline gem that is used to dynamically build the image_science native library) you won't need to provide additional flags in order to locate the FreeImage library.

FreeImage and ImageScience are used in the Apache Olio Rails applications

Our group also recently integrated WordPress into /contrib (as 'wordpress'). Installing this package will pull in all of the Apache and PHP packages required to run WordPress. It leaves the DB to you as you might chose to run MySQL locally or on a completely different system. It actually really makes installing WordPress and getting up and running really easy. We are also working on adding Drupal, Moveable Type, Joomla!, SilverStripe, Cacti, Ganglia, Nagios, the GNU Linear Programming Kit and Octave.

Posted at 11:46AM Sep 25, 2009 by MandyWaite in Open Source  |  Comments[0]

Rails 2.3.3 in OpenSolaris /contrib repo

A while back an OpenSolaris repository called /contrib was introduced to basically allow anyone with an account on opensolaris.org to submit Open Source applications/libraries to an OpenSolaris repo. The details of how to go about doing this are best saved for another blog entry (add to ToDo list). This new repository gave us a real opportunity to package up Ruby on Rails and make it available to OpenSolaris users in the package format that they are familiar with. We'd avoided the main /release and /dev repositories because Rails rev'd too frequently for us to be able to keep up with the changes given that the processes for getting into those repositories are fairly lengthy. Besides there was always the 'gem install' command.

So we've now added Ruby on Rails 2.3.3 and it's dependent Gems to the /contrib repository. Since then Rails 2.3.4 has been released, more on that later.

To install the rails package on a system with OpenSolaris 2009.06 or later you need to run the following commands:

% pfexec pkg set-publisher -O http://pkg.opensolaris.org/contrib  contrib

% pfexec pkg refresh

% pfexec pkg install ruby-rails

% pfexec gem install rack

The first command gives you access to the /contrib repository and all of the packages that it contains and is worth doing anyway as there's lots of great packages there besides Ruby on Rails. It's a one off command, once you've added /contrib it will remain in your list of package publishers. The second command isn't strictly necessary the first time, but it's useful to remember as /contrib updates on a weekly basis and your local cache of package names and versions doesn't refresh automatically.

The next command installs the 'ruby-rails' package and if it's not already installed, installs the SUNWruby18 package which contains Ruby 1.8.7 and RubyGems 1.3.1 (versions correct at the time of writing). The last command installs rack, which is now required by Rails, something we didn't find out until the last moment we moved to Rails 2.3.3. We are looking to package rack and add it as a dependency to the ruby-rails package. The ruby-rails package installs to what is effectively the vendor gem location in /usr and the rails and rake executables install to /usr/bin (as symbolic links). You can still use the gem command to install Ruby on Rails versions later than 2.3.3 if you need to and the /usr/bin/rails command will pick up the later Rails version.

At the moment the benefits of using 'pkg' to install Rails over using 'gem' are mainly just the convenience of having dependencies pulled in automatically and having the 'rails' command on the PATH. In the future, we'll add native gems that are used to provide infrastructure to Ruby on Rails, gems that usually require a compiler to build, along with knowledge of the location of any dependent libraries. Ultimately we'll be able to provide a single package that installs a complete, optimised, Ruby on Rails infrastructure.

Posted at 11:25AM Sep 25, 2009 by MandyWaite in Ruby (on Rails)  |  Comments[0]

EventMachine and the elusive emptys

Having built Ruby 1.9 packages for Solaris Nevada we had to try installing the commonly used gems, particularly those for the Rails stack and specifically the ones we use with the Apache Olio Rails app. One of these was eventmachine which is used by Thin and which is a great piece of Software. It installed fine on our Solaris Nevada b116 x64 box but failed to install on our Nevada b116 SPARC box. The error was pretty bizarre too:

usr/ccs/bin/as: error: no input filename given
usage: /usr/ccs/bin/as [-V] [-Q{y,n}] [-q] [-s]
        [-S] [-K {pic,PIC}] [-o objfile] [-L] [-T]
        [-P [[-Yc,path] [-Ipath] [-Dname] [-Dname=def] [-Uname]]...]
        [-m [-Ym,path]] [-n] [-ul] [-xF]
        [-m32] [-m64]
        [-xarch={v7,v8,v8a,v8plus,v8plusa,v8plusb,v9,v9a,v9b,sparc,sparcvis, sparcvis2,sparcfmaf,sparcima}]
        [-xcode={pic13,pic32}] file.s...
*** Error code 1
make: Fatal error: Command failed for target `binder.o'

What looked suspicious here, was the build line, which looked like this:

g++ -I. -I/usr/ruby/1.9/include/ruby-1.9.1/sparc-solaris2.11 -I/usr/ruby/1.9/include/ruby-1.9.1/ruby/backward -I/usr/ruby/1.9/include/ruby-1.9.1 -I/var/ruby/1.9/gem_home/gems/eventmachine-0.12.8/ext -DBUILD_FOR_RUBY -DHAVE_RB_THREAD_BLOCKING_REGION -DHAVE_TBR -DHAVE_WRITEV -DHAVE_WRITEV -DOS_UNIX -DOS_SOLARIS8 -DWITH_SSL -I/usr/sfw/include -fPIC -g -O3 -fPIC  -g -obinder.o -c binder.cpp

There's no space in '-obinder.o' which you'd normally expect to read '-o binder.o'. It looked odd, though it's not unusual for spaces to be ommited when passing names to options, I'd just never seen it on a build line before so it stood out. Looking at the output with -v set as one of the flags passed to g++, gave more data but not anything useful. We looked at it on the x64 system (again with -v as a flag to g++) and it went through the same steps but didn't fail.  But it did use a different assembler! gcc/g++ on x64 uses /usr/sfw/bin/gas but on sparc uses /usr/ccs/bin/as. It also became apparent by running the assembler manually that it was choking on -obinder.o, it seems that gas is more tolerant than as :o(

The Makefile that gets generated by extconf.rb when you install the gem has lines in it's .SUFFIXES section of the form:

.cc.o:
        $(CXX) $(INCFLAGS) $(CPPFLAGS) $(CXXFLAGS) $(COUTFLAG)$@ -c $<

and earlier in the file COUTFLAG is defined as:

COUTFLAG = -o

Which ultimately results in -obindir.o (same for all of the other objects it builds).

Matz seems to have hit this problem before, because the Ruby 1.9.1 Makefile contains a fix for a similar problem in Ruby (although it still happens when building the ext/*.c sources)

empty =
OUTFLAG = -o $(empty)
COUTFLAG = -o $(empty)

Which results in a space being inserted after the '-o' for $(OUTFLAG) and for $(COUTFLAG)

The Makefile is of course generated by extconf.rb which gets it's instructions on how to create the Makefile from mkmf.rb which Ruby provides. So the fix would be to make the changes outlined above in lib/ruby/1.9.1/mkmf.rb or to have it create the SUFFIXES lines with a space between the COUTFLAG variable and the output file name. We can do that for any packages that we build, but we also need to raise a bug with Ruby.

Posted at 07:02PM Jun 19, 2009 by MandyWaite in Events  |  Comments[1]

Building Ruby 1.9.1 on Solaris SPARC

We've been working on building Ruby 1.9.1 (p129) packages on Solaris Nevada (the platform that OpenSolaris is mostly built from). We hit a couple of problems on the way, one was easy to fix the other not so.

The first issue was Sun Studio borking when it found a function declared with a return type of void but which actually contained a return statement. gcc actually thinks this ok which seems odd to me, maybe it's just our version of gcc though... When gcc hits this it says:

pty.c:425: warning: `return' with a value, in a function returning void

But with Sun Studio cc you get:

pty.c:425: void function cannot return a value
cc: acomp failed for pty.c

The file causing the error is ext/pty/pty.c and line 425 has the offending line in function getDevice() (declared as 'static void'):

return Qnil;

If you comment out this line it will build ok. There is a bug for this issue......

The other problem only occured on SPARC initially, we were using one of the public build systems that we have for the SFW consolidation (the part that delivers most of the F/OSS into Solaris Nevada and OpenSolaris) and that was running Solaris Nevada build 114, for x64 we were building on Solaris Nevada build 116 which was the latest version available at time of writing. x64 built fine.

The error seen on SPARC was:

Undefined                       first referenced
symbol                             in file
rb_cFalseClass                      enc/emacs_mule.o  (symbol scope specifies local binding)
rb_cTrueClass                       enc/emacs_mule.o  (symbol scope specifies local binding)
rb_cFixnum                          enc/emacs_mule.o  (symbol scope specifies local binding)
rb_cSymbol                          enc/emacs_mule.o  (symbol scope specifies local binding)
rb_cNilClass                        enc/emacs_mule.o  (symbol scope specifies local binding)
ld: fatal: symbol referencing errors. No output written to .ext/sparc-solaris2.11/enc/emacs_mule.so  

when linking enc/emacs_mule.so. It's not a well documented error, but the implication was obvious, the required symbols had been found but they had been declared as local and so couldn't be used to build this shared object. Using nm on emac_mule.o and on libruby.so.1 seemed to indicate that the symbols were needed by emacs_mule.o (UNDEF) and were available in libruby.so.1. We asked the compiler experts and they thought that perhaps the symbols were declared as HIDDEN. We tried elfdump on both emacs_mule.o and libruby.so.1 and guess what, when analyzing libruby.so.1, elfdump threw up loads of errors of the type:

"bad symbol entry: <address> lies outside of containing section"

This suggested that the shared library was broken in some way.

We isolated the linker lines from the build for libruby.so.1 and ran them individually (after touching ruby.c). There were two lines, the first was the linker line which actually built the shared library. That ran ok and when we ran elfdump on the resultant library there were no errors. The second line was:

/usr/sfw/bin/gobjcopy -w -I "Init_*" libruby.so.1

After running this manually we saw the same error when using elfdump on the resultant library.

At the same time as this we were running a build on a SPARC system that we'd had upgraded to Nevada build 116 and that completed OK. A check on the version of gobjcopy on the two systems showed that we had gobjcopy 2.15 on the build 114 system and 2.19 on the build 116 system. Further checking showed that gobjcopy was delivered into Solaris Nevada in SUNWbinutils and that had been updated in Nevada build 116. So the problem wasn't the fact we were building on SPARC but that we were building on different OS revs, the problem also exists on x64.

At the moment we haven't looked into what was going wrong when gobjcopy tried to make the Init_* symbols local, but it was apparently corrupting the library.

At the moment this makes it tough to build Ruby 1.9 on OpenSolaris which is based on Nevada build 111b and we are looking at how best to get around this. Maybe make the packages available from the /webstack repository. In the meantime we'll file a bug against OpenSolaris and come up with a workaround.

Posted at 06:15PM Jun 19, 2009 by MandyWaite in Events  |  Comments[0]

Apache Olio: Web 2.0 toolkit for Java EE

This week saw the Apache Olio project release the code for it's Java EE version adding to the versions already available for PHP and Rails. 

If you know Apache Olio and want to know more about the specifics of the Java EE version then I'll cover them first, if you want to know more about Apache Olio in general, read on.

To run the Java EE version of Olio, you'll need:

• Java SE 5 or 6
• Java EE 5 compliant application server (tested so far with GlassFish v2)
  
• A Java Persistence API (JPA) provider (Eclipselink is the JPA provider packaged with GlassFish v2)
• MySQL Database (any DB could be used but we have scripts and instructions for MySQL)

Some of the technologies that the Java EE version features:

• JPA for the Object-Relational persistence solution
• AJAX 
• Rest based Services (partially implemented)
• JMaki widgets wrappers for Yahoo and dojo widgets

In planning are the following changes/features:

What is Apache Olio?

Apache Olio is a Web 2.0 toolkit, basically it's a Web 2.0 application and a load generator. You deploy the application to a configuration that you want to test, fire up the load generator, drive load to the application and then analyze the results. The application isn't that fussy about what it runs on, for the Java EE app you need a Java EE Web container (Glassfish or Tomcat for example), you also need a Database and a schema is provided that can be used to set that up. You need a filestore and you need a Web Server to act as a remote Web Service (for looking up geolocations). 

Apache Olio uses Faban to drive load along with a custom Faban driver. Faban is a benchmark driver framework and harness that is designed to allow you to model the usage of your application and drive load for 1000s of simulated users. It also can be used to manage the runtime environment and it gathers the results from test runs.

Once you've deployed the application you can load it up with dummy users and events (it's a Social Networking app) and use the driver to simulate load. At the end of a test run, you get all of the data from the run presented to you in graphical form (depending on the platform). I spend a vast amount of time using Olio and Faban and can't recommend them enough.

Kim is the lead developer of the Java EE version of Apache Olio and he has a blog entry that goes into lots of detail on how Apache Olio Java EE works and what it looks like.

Posted at 05:04PM May 14, 2009 by MandyWaite in Open Source  |  Comments[0]

Manic Street Preachers new album

Anyone that knows me wouldn't be surprised to learn that I identify with (and still love) Richy Edwards of the Manic Street Preachers. Richey has been missing for 14 years now but his lyrics live on, particularly in the Manics greatest work "The Holy Bible". I'd heard rumours last year that the manics were going to make a record similar to THB but I snorted with derision given that Richey couldn't be part of it. Well surprise, surprise, they have made the album and using lyrics that Richey wrote but which were never recorded. I've heard the opening track "Peeled Apples" and it's very, very good, very reminiscent of THB. It's kind of amazing that James Dean Bradfield (singer and lifeforce of the Manics) still has it in him to sound so very, very angry :o)

The album is called "Journal for Plague Lovers" and is released on May 18th. Track listing is:

Peeled Apples
Jackie Collins Existential Question Time
Me and Stephen Hawking
This Joke Sport Severed
Journal For Plague Lovers
She Bathed Herself In A Bath Of Bleach
Facing Page: Top Left
Marlon J.D.
Doors Closing Slowly
All Is Vanity
Pretension/Repulsion
Virginia State Epileptic Colony
William’s Last Words

I have real hope that this will be the album that they would have recorded after The Holy Bible if Richey had still been with them. Because of this I've booked tickets to see them on May 28th May at the Roundhouse in London in the hope that I'll love the album and that at the gig they'll play a mix of tracks from the new one and from THB and maybe for once not play the tedious track they seem to be most famous for "A Design for Life".

I actually saw Hawkwind at the Roundhouse believe it or not, I was very young and lived in Camden Town. So going back there will be a blast. I'm also going to see Andrew Bird at the Thekla in Bristol with my Friend Michelle on the 10th May.

4Real

Posted at 02:52PM Mar 27, 2009 by MandyWaite in Personal  |  Comments[2]