As you can see, this blog has been dormant for quite some time. However, I started up a third blog attempt several months ago at owenkellett.com. Just thought I’d let anyone out there know…

Recently (just completed today), our group has migrated from Perforce to Subversion for source control of all of our automation test scripts and harness code. The decision to make the move was largely more for political and economic reasons than technical ones, but personally I do feel that for our specific needs, Subversion is a better overall choice. The advantages that we gain include:

• Cost. Perforce costs roughly $200 per user per year. With our group holding upwards of 30 licenses, this becomes a non-trivial cost. Subversion is open source and free. The benefit here is clear.
• Easier code sharing and collaboration between us and other internal Sun groups. Seeing as many of our tests can be used by other QA groups within Sun testing similar or identical products, Perforce licensing hurdles have started to block some of our collaboration initiatives.
• More flexibility when editing checked out code. One of my personal pet-peeves of Perforce is its requirement that you explicitly notify the repository that you are going to make a modification on a file before it allows you to modify it (i.e "p4 edit"). Not only does this prove to be a minor annoyance while editing files, it also makes it difficult to work on checked out code when you do not have direct network access to the repository. I do realize that there is a workaround for this problem, and I also realize the benefits of notifying the repository when code is being edited. In general, though, it makes the overall experience doing even simple tasks in Perforce cumbersome and somewhat unintuitive.
• Consistency between us and Andromeda/Galaxy developers as well as other Sun QA groups. In general, it seems that Subversion is the source control system of choice for many of the groups with which we are directly involved. Using the same system only make sense.

As far as disadvantages go, the one main pitfall that has been touted in comparisons between Subversion and Perforce is that Subversion does not have automatic branch tracking. Thus fundamental software engineering techniques of maintaining multiple branches of code and tracking and merging changes between branches is a considerably more manual and error-prone process in Subversion. However, since, as a QA group, we utilize merging features considerably less than a software development group would, I would contend that this negative has minimal impact for our purposes.

The move to Subversion is therefore a welcome one. With it, though, comes the minor overhead of adopting slightly different techniques when working with a Subversion repository as opposed to a Perforce depot. For the benefit of all, here are some basic tasks that many members of our group are used to doing with Perforce, along with their Subversion equivalents [Note these are guidelines for the Unix-based clients. I stay away from Windows as much as possible.]:

• Checkout code from repository:
  
  Perforce: The process of setting up a local workspace in Perforce always seemed overly complex to me. You had to set multiple environment variables (usually P4USER, P4CLIENT, P4PORT, but others configurable) to configure the repository location and user credentials. You then needed to setup a client workspace, mapping depot directories to local directories. After this process, "p4 sync" would check out a copy of the desired workspace for you.
  
  Subversion: Simpler.
  svn checkout --username [USERNAME] http://repository/location/and/desired/subdirectory [local directory]
  
  
• Update your local copy to the latest revision in the repository:
  
  Perforce:
  p4 sync
  
  Subversion: One step process.
  svn update
  
  
• Make changes to a file in the repository:
  
  Perforce: Two step process.
  p4 edit [filename]
  Modify the file as desired
  
  Subversion: One step process.
  Modify the file as desired
  
  
• Add a new file to the repository:
  
  Perforce:
  p4 add [filename]
  
  Subversion:
  svn add [filename]
  
  
• Delete a file from the repository:
  
  Perforce:
  p4 delete [filename]
  
  Subversion:
  svn delete [filename]
  
  
• Undo changes to a file in your local copy:
  
  Perforce:
  p4 revert [filename]
  
  Subversion:
  svn revert [filename]
  
  
• Check the status of modified, added, deleted, etc. files in your local copy:
  
  Perforce:
  p4 opened
  
  Subversion:
  svn status
  
  
• Compare a modified local copy of a file to the original checked out copy:
  
  Perforce:
  p4 diff [filename]
  
  Subversion:
  svn diff [filename]
  
  
• Commit changes from your local copy into the repository:
  
  Perforce:
  p4 submit
  An editor window using the program specified by the environment variable P4EDITOR will open for you to enter a description of the changes.
  
  Subversion:
  svn commit
  An editor window using the program specified by the environment variable SVN_EDITOR will open for you to enter a description of the changes. Alternatively, you can bypass the editor and enter the description inline:
  svn commit -m "This is a description of the changes"
  
  
• Create a branch:
  
  Perforce:
  p4 integrate [source_tree] [dest_tree]
  Perforce automatically keeps track of branched files. Therefore, merging changes between branches is more automated.
  
  Subversion:
  svn copy [source_tree] [dest_tree]
  As far as SVN is considered, once a tree is branched, they are completely separate and independent copies. After branching a tree, it is good practice to make note of it in the revision log so that the revision number can be found later when changes need to be merged with other branches.
  
  
• Move a file:
  
  Perforce: Two steps.
  p4 integrate [source file] [dest file]
  p4 delete [source file] In order to move a file in the Perforce depot, the source file must be branched and then deleted. This creates a somewhat counterintuitive revision history since instead of associating the entire revision history log with the new file, it simply points to the revision history of the originally branched file and maintains all history from that point forward.
  
  Subversion: One step.
  svn move [source file] [dest file]
  Under the hood, this performs the same copy, delete actions as the Perforce process. However, with SVN the entire revision history is copied to the new file since it does not maintain automatic branch information. For simple file moves, this is a more desired behavior.
  
  
• Resolve conflicts between your local changes and others' conflicting changes:
  If changes to certain files have been made since you last ran an "svn update", any conflicting changes that you have made to common files need to be resolved.
  
  Perforce: Semi-automated process.
  p4 resolve [file]
  Perforce will walk you through all of the conflicts between your changes and the others' changes and ask you how you would like to merge the changes. There is also the option to resolve the conflicts manually.
  
  Subversion: Completely manual process.
  After conflicts have been discovered (either when trying to do an svn commit or after an svn update), Subversion leaves three copies of the file in your local tree: filename.mine, filename.oldRev, filename.newRev. These represent your new copy of the file, the original checked out copy of the file, and the repository's current copy of the file respectively. At this point, it is up to the user to examine the differences between the three files and decide how to merge the conflicts together. After the file is updated to the user's liking, the following command must be run:
  svn resolved [filename]
  At that point, the newly resolved file can be committed to the repository.
  
  

There is a lot more that you can do with both of these source control systems, but for basic needs, this list is fairly comprehensive. An excellent online book that documents the complete feature set of Subversion can be found here: http://svnbook.red-bean.com/ and Perforce docs can be found via http://www.perforce.com

The dust is still settling from this morning's announcement. Aside from the story being picked up by every tech news site out there, it also unleashed a blogging storm on both blogs.sun.com, and the rest of the blogosphere. Yes I'm talking about Sun open sourcing Java (under the GPL no less).

Not to be left out, let me just say that this is absolutely the right move and will prove to be a win-win-win for Sun, the Java community, and the entire open source community (all of which are not mutually exclusive). Personally, I think the biggest gain is summed up in this piece from Jonathan's blog:

The GPL is the same license used to manage the evolution of GNU/Linux - in choosing the GPL, we've opened the door to comingling the communities, and the code itself.

The whole idea of Java grew out of the "write once, run anywhere" mentality. However, when the platform used to run applications is a closed, proprietary system, conflicts of interest crop up everywhere. Granted, the JVM was always free, and the source code was never really secret, but licensing incompatibilities restricted some groups (such as many Linux distributions) from bundling Java as part of its system. Thus there was a wall separating two otherwise completely compatible communities.

Today Sun has taken down that wall. It is now up to the open source community to explore the potential benefits that can be gained.

It's been a week since I sprained my ankle during the ultimate game last Friday and the good news is that it has shown some significant improvement. I've progressed to doing light workouts on it along with suggested rehabilitation stretches and exercises. I can walk without any limping or pain and minimal discomfort. It's still sore, and the swelling has still not completely gone down, but I'm managing it with ice and it's improving each day. The hardest part, though, is staying off of the field. I was tempted even today to go out to the game but I resisted, even though I'm young and invincible :). I'm going to try to give it at least another week before I lace up the cleats again.

This is the first time I will have gone more than a week without playing a game of ultimate in over a year. Do they have a patch or chewing gum for this?

On the political spectrum, I would probably best describe myself as a conservative Democrat. On one hand, I do, for the most part, believe in a free market economy. I do believe that capitalistic ideals have their place in society, and that those who are ambitious, opportunistic, and even lucky deserve to be rightfully compensated. Unfortunately, though, greed and indifference often drive people to overextend this ambition. Once one's (or many's) accumulation of wealth comes at the expense of the health and well-being of less fortunate people and their families, it is no longer acceptable. Certain industries, such as health-care and education should be funded in full by the government so that all citizens are at least afforded the fundamental life-sustaining services without regard to economic status.

I know that I am speaking in idealistic terms, but how can progress be made unless we set far-reaching goals? How can we possibly hope to improve ourselves if our leaders promote a vision of the future that is riddled with doomsday scenarios and fear-mongering? How can we create a better life for Americans if our civil liberties are stripped away in the name of "national security." Fear is not the way of the future; hope is. We must always believe that tomorrow will be a better day. Get out and vote.

This blog has been up for less than a week and I've already been hit by my first comment spammer, Mr. "xvsasd"!. Is that even possible? I suppose since all Sun blogs are automatically syndicated to the blogs.sun.com feed, that's how links to my posts are being picked up. I guess I'll be manually moderating my comments for now. *sigh*

A few months back, I took on co-system administration duties for one of our groups most critical data and application servers. It provides several important services, including housing our back-end test harness database, maintaining the Bugzilla database for Sun's Secure Application Switches, and acting as the main execution server for our test automation. Until recently, it was cranking away on a five year old system running an ancient version of Red Hat (7.1 I believe). In desperate need of an upgrade, we got a hold of an X4200 to replace it.

The X4200 class of servers is certified to run Solaris, Red Hat Enterprise Linux, SUSE Linux Enterprise Server, and various editions of Microsoft Windows Server. Flying in the face of logic, we chose to install Debian stable (along with the custom kernel compiling that was involved to provide support for the X4200's SAS disks). There are a lot of advantages to running a Debian system, most notable its superior apt-get package management system. After some initial aggravation setting it up, it has performed with minimal complaints since.

Our situation is somewhat unique, however. We have just one system to support with only a small set of functions. Most corporations, though, would not choose to buy racks of expensive Sun hardware and then run an unsupported operating system on them. Those who want to take advantage of the benefits of a Debian-based distribution are on their own. It appears, though, that this may soon change. According to reports, Sun and Canonical are ready to announce plans to certify Ubuntu (a Debian derivative) on Sun's x86 servers (including X4200's).

Yet another step in the right direction.

Argh. It's a given that injuries are a part of participation in any sport and I have dealt with my fair share of them. Various repetitive stress and overuse injuries cropped up during my track and field career in high school and college. Regardless of how many injuries I've dealt with, though, each new one always seems to be equally as aggrevating.

You can probably see where I'm going with this. I sprained my ankle during yesterday's ultimate frisbee game - landing on the side of my foot rather than the bottom of it after going up for the disc. It wasn't a major sprain. It initially didn't swell up at all and I continued to walk around on it with some annoyance during the rest of the day. When I got home late last night, though, I took off my shoes and noticed that one ankle was definitely bigger than the other. After some ice and a night of sleep, I woke up this morning with a very stiff and still somewhat swollen ankle. Awesome.

So I'm on the sprained ankle recovery program: minimize weight-bearing activities, and R.I.C.E. treatment. I've got ice wrapped around it right now, and I'm going to re-apply it every two to three hours or so today to see if I can get the swelling down. I've also taken some ibuprofen and have my legs propped up so that they are elevated. What's the hardest part though? Staying off the field until I'm fully recovered. I've been playing lunchtime pickup ultimate games Mondays, Wednesdays and Fridays since I started working here over a year ago last October. In that time period I've rarely missed a game, and haven't gone more than a week straight without playing. I clearly won't be able to play next week, but it's probably not wise to play for at least a week or two after symptoms are gone and I've properly carried out a stretching and strengthening exercise program.

I hate being injured. Somebody hold me back.

One feature that is sorely lacking in our test automation harness is the ability to share and lock resources between competing groups of tests. Consider this situation: Our group is responsible for testing some of the systems in Sun's X64 family of servers. In particular, much of our focus is on the Sun Blade 8000 (code named Andromeda). Given two indisputable facts:

1. These systems are expensive, and thus there are not an infinite supply for us to test.
2. There are many different components to one system, some of which can be tested independently.

It is possible (and in fact very likely) that we might try to run competing groups of tests on the same hardware at the same time. The reality is that we've run into this problem a lot more than we care for, and our current test automation harness provides no mechanism to ensure that separate groups of tests do not collide in this manner.

Thus we are faced with a somewhat classic resource sharing problem, but with a unique twist: some resources (such as an Andromeda chassis) might have sub-resources which can be locked and used independently and without interference. Therefore, a resource can only be locked for use if it and all of its sub-resources are available. Our specific problem details can be described as follows:

• Resources are structured in a hierarchical format as just described.
• The resource hierarchy is stored in a centralized SQL database.
• Each logical group of automated tests defines a set of resources that must be locked before execution can begin.
• Each group of automated tests is run by an execution daemon. There may be multiple execution daemons running on possibly different servers which will be attempting to lock the same resources in the centralized database before executing the tests.

One of the considerations in coming up with a resource sharing policy for this situation is that we should strive for simplicity over any optimizations made for speed or to maximize concurrent execution. While both of these benefits would be nice, the number one priority is to ensure a consistent, well-known, and repeatable test environment so that the test results are not invalidated by external interference. Any complexity that we add to the design introduces additional possibilities for infrastructure problems which could create overhead in test maintenance and debugging. Throughput priorities should be secondary which is why I have enforced the policy that all resources required for the entire life of a group of tests (even if it is only required for a short period of time or only a few test cases) must be locked by an execution group before any tests in that group can begin.

It turns out that having a hierarchy of resources does not present a significant problem in coming up with a resource sharing policy. As long as resources and their children are properly ordered per Dijkstra's solution to the infamous dining philosopher's problem, and resource locks are always acquired in the same order, the same basic principles apply. Thus here is my basic locking algorithm presented as a solution to the above described problem:

• An execution group requires a set of resources, we'll call them R
• Foreach resource r in R (in alphabetical order)
  • Lock the database tables associated with resources (to avoid race conditions in database access).
  • If r is unlocked AND there is nobody waiting in the queue for r (the queue is maintained in the database), lock it (by assigning your name to the resource as the locker in the database).
    • Recursively lock each child resource of r in alphabetical order.
    • If any children of r are locked, release all children of r (but do NOT release r itself). This will allow any execution groups that require only children of r to execute while we are waiting for resources to be released. After waiting n (10?) minutes, go back to the beginning of the children locking loop and re-attempt to lock the children. This time, do NOT release the child locks if a locked child is encountered, enter into the queue for that child instead and initiate the pseudo-busy waiting process for that resource.
  • Else enter into the queue for resource r and initiate the pseudo-busy waiting process for r.
  • Unlock the database tables associated with resources
• Once all resources are acquired, execute all tests in the group and then release the locked resources in R
• Pseudo-busy waiting process:
  • sleep for (5?) minutes.
  • check if the resource is available. If it is, lock it and remove yourself from queue. Otherwise, go back to the previous step.

As you can see, the algorithm is not built for speed and is decidedly greedy in its locking policy. Like I said above, though, speed and concurrency are not the main goals. Reliability, consistency, and simplicity are. There are still a few problems to iron out, but this basic algorithm should ensure that groups of automated tests contending for the same resources do not interfere.

So I've decided to give it another shot. My last blogging attempt (along with it's short-lived corollary, my internal Sun blog) was sustained for a while, but quickly faded after I started working here at Sun Microsystems. After much contemplating, though, I think I'm at a point where I can give it another go.

With this decision made, the first order of business was to decide on the location of my Blog2.0. I did consider reviving owenkellett.info, but ultimately I decided to leave it alone and go a different route for a few main reasons:

• As an employee of Sun Microsystems, I do feel somewhat compelled to contribute to the corporate blogging effort that Sun, and in particular CEO Jonathan Schwartz has pioneered.
• owenkellett.info was started before my first full-time employment and has a very limited overlap with my current tenure at Sun.
• I felt the need to start fresh but do not yet want to purge my previous blog's identity.

Considering all of this, I was left with an obvious option: launch a new, public, personal blog as part of the blogs.sun.com community. My only apprehension about this prospect, though, was that Sun's blogging site uses Roller as its blogging engine while I had become particularly attached to Wordpress during my owenkellett.info toils. Having previously explored Roller, I was not left particularly impressed.

It turns out, however, that Roller, while it may be behind in a few features and have less-robust plugin support in comparision to Wordpress, suffers mostly from its smaller user base and less established development community. Thus while Wordpress has a large collection of community developed themes and plugins, Roller has a much smaller set. In addition, it may just be a matter of personal preference, but I don't like any of the stock Roller themes installed for blogs.sun.com. To be vague, they all feel very "Web1.0" as opposed to belonging to the blogsphere "Web2.0" style.

So last night I dug through the Roller docs and cracked open a can of css to see what Roller can really do. I have to admit that I am a fan of the Java/Hibernate combination which makes up the foundation of the back-end engine. Additionally, the use of the Velocity template language to generate the web gui front-end is pretty slick and makes for fairly clean, modular, and simple theme and template code. And while I don't claim to be a css expert by any means, the result of my efforts is the layout of this site. In my opinion, it looks more "blog-like" than any of the stock templates in the blogs.sun.com library. [Although since I do not have a machine with Windows installed anywhere that is readily accessible, I still have not been able to test the layout on Internet Explorer. The structure is fairly basic though, and to the best of my knowledge W3C standards compliant.]

In any case, I'm back. Hopefully I can stick with it this time, and that this blog will help me to keep my perspective on whatever lies ahead.