The SourceJuicer Blog

Many thanks to Emanuele, Andras, Thomas, Michal and everyone else who attended and helped with OSDEVCON 2009.  And thanks to Dirk and the German Unix Users Groups who organized this event!  I appreciated the comments and great ideas for improving sourcejuicer.  

I'll be presenting OpenSolaris SourceJuicer at the OSDevcon conference in Dresden this Thursday, October 29th at 3:00p.m.  I hope to see you there!

I'll be hosting a Birds of a Feather (BOF) at OSCON 2009 entitled:

OpenSolaris Source Juicer: Discussing Cloud-based Community Software Development

It has been scheduled for 7:00 p.m. on July 22nd in Ballroom A3/A6 at the McEnery Convention Center in San Jose California.  Attendance of the expo hall and BOF sessions only is free.

I hope to see you there!                                 ---Brian

Jim Walker pointed me to Markus Weber's screencast demonstration of OpenSolaris SourceJuicer.  If you have a recent version of Adobe Flashplayer installed, you can view it here:

I used Alex Shindich's Apache Log Analyzer for a quick look at Source Juicer's logs to see whether Dermot and Christian's demonstrations at community one are having an impact.  I think I see the beginnings of a nice trend:

A new release of OpenSolaris Source Juicer is now live on jucr.opensolaris.org.

Release 1.2.0, contains a large number of bug fixes and enhancements, including UI improvements and back-end Build Grid fixes.  Here are some of the bug fixes:

ID	Summary
8795	submission time file name validation
9001	IPS packages not published for successful builds
7530	Delay in the data being displayed in review thread
7244	Add new comment/vote/re-submit box to bottom of review thread
8797	Build Status reporting FAILED for successful build
6240	Submitted spec files do not appear on 'Build' page
7246	Parse spec at submit time for validation info and syntax check
7957	Describe BuildGrid environment
8771	UI - Help page - Add more spec,sj & FMRI
8002	Check box for Sun employees OSR requirement on submission page
7933	Pagination: sorting between pages does not work
9092	Add p5i link to build status page instead of FMRI
9091	Need to improve opensolaris.org and source juicer logos in header
7922	Add pagintation to review and build pages
8886	blogs.sun.com/jucr link points to wrong url
8692	UI - Make diagram on homepage active & point to workflow description
8691	UI - Homepage should not need scrolling
8736	UI - Review tab - Remove total votes field
8856	Rendering incorrect in Safari and Internet Explorer
8968	MyBuilds empty because submitter not associated with build
8462	Implement sorting on build status page
9083	Broken hyperlink to spec file help page
6511	Error page when user who is not logged in selects 'upload a package'
8930	Redirected to Review instead of Home after logout
6910	comment-formatting should be retained

You may also notice some improvements in build grid and UI enhancements such as:

• Review and build status pages are paginated
• Identifier, Submitter, Build Start, and Build Finish fields are populated on build status page.
  
• Columns in Review and Build status pages are sortable
  
• Some fields were added and some removed to make build status and review pages more efficient.
• One click installs!  (An FMRI link was was added to the Install field in the Build Status page.  If your build worked, click on it's install field and opensolaris will launch package manager and ask if you want to install this package.)
  
• The submit page now makes it obvious that spec file and copyright are the two necessary file uploads. 
  
• The submit page also does some checking on the identifier and file names, enforces naming convention and doesn't let you contribute packages entitled SUNW{foo}.
  
• Some UI enhancements were also added to the main page, Submit and Myjucr page.
• More help content was added.
  

Please let us know if you have any requests for additional enhancements or log bugs if you find them.  We hope you enjoy working with the new OpenSolaris Source Juicer!

We've had a few queries on how to set up your own build environment that duplicates the one on jucr. Well, here it is. This is all stolen from mattman's blog.

Install dependencies

pfexec pkg install ss-dev SUNWgnome-common-devel SUNWperl-xml-parser \
                   SUNWgnome-xml-root SUNWgnome-xml SUNWgnome-xml-share \
                   SUNWdoxygen 
pfexec /usr/share/sgml/docbook/docbook-catalog-install.sh

Get desktop CBE and install it

wget http://dlc.sun.com/osol/jds/downloads/cbe/test/desktop-cbe-1.7.0-rc1-x86.tar.bz2
bunzip2 desktop-cbe-1.7.0-rc1-x86.tar.bz2
tar -xf desktop-cbe-1.7.0-rc1-x86.tar
cd desktop-cbe-1.7.0
./cbe-install

Remove pkgbuild and install newer version

pfexec pkgrm SFpkgbuild
cd ..
wget "http://prdownloads.sourceforge.net/pkgbuild/pkgbuild-1.3.98.4.tar.bz2?download"
cd pkgbuild-1.3.98
./configure --prefix=/opt/dtbld/
make
pfexec make install
⁞

All done!

1. Add the repo to your authority list. You only need to do this the first time you install a package, obviously.

$ pfexec pkg set-authority \
    -O http://jucr.opensolaris.org/pending jucrpending

2. Refresh

$ pfexec pkg refresh

3. Install the package you're interested in.
   

$ pfexec pkg install <my-package>

Easy!

If you've been trying out jucr, you may have noticed that your packages are not appearing in pkg.opensolaris.org/pending, but rather jucr.opensolaris.org/pending. So, what's the reason for this? At the moment, there is no facility in IPS to authenticate a publisher, as in, someone sending a package to the repo. This means that any web facing repo must be run in read-only mode. This is the case with 'official' repos (pkg.opensolaris.org/pending and pkg.opensolaris.org/contrib). So, in order to get a package into either of these we need to:

1. Publish to an invisible read-write copy of the repo
   
2. Stop the read-write copy
3. zfs snapshot the filesystem
4. Halt the 'real' repo
   
5. zfs send | zfs recv an incremental to the machine hosting the 'real' repo
6. Re-start the 'real' repo

Which, as you can imagine, is something you don't want to do very often. Seeing as pushing to /pending will be frequent and pushing to /contrib infrequent, we decided to run a /pending repo ourselves but continue to use the official /contrib. We can do the zfs snapshot trickery a little more easily (the repos will be on the same machine). Plus, users will tolerate a delay for pushing packages into /contrib, but less so for /pending.

The SourceJuicer team has announced that SourceJuicer 1.0 is now available! SourceJuicer is a web service which facilitates the building and review of OpenSolaris packages. Its goal is to pave the path between building an OpenSolaris package, (which may only work on your laptop) and publishing high quality packages into the OpenSolaris /contrib repository. Here are some SourceJuicer features:

• It preinstalls the CBE environment in a build zone
• It provides a community package review system.
• It provides examples and automatic validation to help walk the contributor through the required spec file fields.
• Once a package passes validation criteria and builds, it appears in sourcejuicer's internal /pending package repository and can be immediately installed on a clean QA machine for testing:

  pfexec pkg set-authority -O http://jucr.opensolaris.org/pending jucr-pending
pfexec pkg refresh
pfexec pkg install {foo}

• If the package is clean and passes community review, it can continue on to the /contrib repository.

We're looking forward to hearing feedback as OpenSolaris contributors make use of SourceJuicer.  You can find out more about the progress of sourcejuicer here on the SourceJuicer Blog.

There are literally hundreds of thousands of open source projects out on the web with an equally daunting number of people actively contributing to those projects. Sourceforge alone has 180,000 projects and 1.9 million registered users. Not only are these people making the code work, they're writing documentation, providing support to users, creating graphic art and so on. So, it's a no-brainer, if you want any sizable number of these projects in your OS, you need help. There's help to be had, though. There are people only too willing to contribute, as long as you provide the tools to let them. Some basic minimum requirements, as I see it, are:

1. Low barrier to contribution
2. Facility to review each others work
   
3. Uniform build system that scales
4. Trusted users
   

SourceJuicer's goal is to provide tools to fit these tasks.

Currently there are two community IPS repositories - /pending and /contrib. The /pending repository is considered to be focussed on developers while the /contrib repository is for users. Any user can push a package to the /pending repository - it just has to pass some basic validation. This push begins a review of that package by the other users on the system through a web app. They can propose improvements and even re-submit the package, including those changes, for further review. 'Trusted' users can vote on the package and, if it gets enough votes, it will be promoted to the /contrib repository. The format of the submission is a spec file. This allows all parts of the package to be reviewed, from source to binary. It also means that we can build the package for you automatically in our build system and publish it to the appropriate IPS repo.

We're currently running a beta program. If you'd like to sign up as a beta tester, just ask on our alias:

http://www.opensolaris.org/os/project/sourcejuicer/discussions/

If you want more info, take a look at the project page, here:

http://www.opensolaris.org/os/project/sourcejuicer/

You can also ask us questions on IRC here:

irc://freenode.net/#opensolaris-jucr