Things on my mind. George Drapeau's Weblog

I'm a sucker for a good multimedia package.  I love playing with software that does movie playback on encoding, audio playback, transcoding...you name it, I love playing with it.  So naturally I was pretty psyched to see that somebody has contributed the Fluidsynth software synthesizer package to OpenSolaris.  Here is what I did to install it and run it in order to prove that the package actually works.  (here's a spoiler for you: it does work.)

Fluidsynth was put into the OpenSolaris "pending" repository, the idea being that people will check it out and if it's deemed to be of reasonable quality, it'll get voted into the contributed software repository.  The "contrib" repo is where good packages go after they've been testing in the "pending" repo staging area.  We haven't voted on Fluidsynth yet as of the time I write this; I'm hoping that will change after people read what I've done here.

(note: for my testing, I am running OpenSolaris build snv_111b)

First things first: set up OpenSolaris to find packages from the Source Juicer "pending" repository with these two steps:

1. type "pfexec pkg set-publisher -O http://jucr.opensolaris.org/pending jucr-pending"
2. type "pfexec pkg refresh"

Next, I launched the Package Manager application and chose the "jucr-pending" repository from the pop-up menu on the right side of that application's user interface.  After Package Manager thought about its new catalog of apps for a moment, I saw a list of hundreds of packages available to me.  I used the search field to type "fluid", and found the "fluidsynth" package.  Selected that, saw that it lists several other packages as dependencies (i.e., those other packages had better be installed for Fluidsynth to work correctly), then I installed it.  Nicely, the Package Manager installed Fluidsynth and its dependent packages for me.

At this point I ran into a little snag, which is not the fault of OpenSolaris: I'm testing this within OpenSolaris, but OpenSolaris is running as a VirtualBox guest on my Mac Book Pro.  Turns out that audio support in OpenSolaris under VirtualBox needs a little bit of work to get going.  It's easy enough, though, and took less than five minutes to get it working.  The instructions on how to make it work are in this blog post, which is clearly written.  And I had fewer problems than were mentioned; I didn't have to reboot or uninstall the "SUNWaudiohd" package.  Good times!

At this point, my big challenge was where to find two files to test Fluidsynth: a sound font (basically, a description of instruments that Fluidsynth uses to play music), and some music (a MIDI file).  I did a Google search to find a nice Yamaha DX-7 electric piano sound font (I happened to find it here), and it was easy to find any number of .mid files to play.

To test, I typed "fluidsynth <name-of-sound-font.SF> <name-of-MIDI-file.mid>".  That worked just fine: I heard the music loud and clear, although Fluidsynth complained that there is no /dev/midi.  I believe it is expecting me to connect a MIDI keyboard to the computer and start playing, which is not necessary for this test.  Also, Fluidsynth had to re-map some of the MIDI file's preferred instruments to what was available in the sound font's instrument library.  Not a problem, though.

Just for fun, I tried turning off the built-in chorus and reverb effects, and I boosted the amplitude to see if these features worked:

"fluidsynth --chorus no --reverb no --gain 0.8 <name-of-sound-font.SF> <name-of-MIDI-file.mid>".  I also tried changing these parameters individually to isolate the effects.  Again, this worked fine.

As far as I can tell, Fluidsynth works perfectly well on OpenSolaris.  It should make a fine addition to the contrib repo.

Powered by ScribeFire.

Here is a link to a press release from the European Commission, announcing that they have cleared Oracle's proposal to acquire Sun Microsystems.  I find it interesting reading because it describes the fundamental issues that the Commission dealt with before making their decision (MySQL, Java, integration of a complete technology stack)

From an open source point of view, I found it satisfying that they mentioned PostgreSQL as part of their deliberation.  They said that they found the PostgreSQL open source database to be a credible alternative to MySQL; essentially, they're saying that the open source DB landscape is not monocultural, and that there are other viable alternatives.  I think they're right, and I'm glad to see that it's not all about MySQL.

My opinion: they took a long time to make a decision, but they considered the issues you'd think they should consider.

The next few days should be really interesting for us at Sun.  Whether employed or not, at least the long purgatory period is finally coming to an end.  I can't tell you how tiresome it has become telling my friends month after month "No real news yet."  It's nice that friends care, but it sucks that there's been no progress to report.  Finally, that's done with.  (whew!)

Powered by ScribeFire.

Now here's an interesting use of live blogging.  This snapcast is basically aggregating tweets and other info related to the Haiti situation following the 7.0 magnitude earthquake that hit there this week.

Check it out.  Also, if you know of reliable Twitter accounts that are tweeting Haiti-related help info (e.g., if a charitable organization is tweeting about how we can help), I'd love to hear about it.  You can post a link as a comment to this blog post.

Powered by ScribeFire.

I've mentioned G-Snap! before on this blog; a buddy of mine left a high-profile job at Yahoo! a couple of years back to start a company around a vision he had about how to use mobile computing to bring people together.  He and his company have been doing some really cool work with their app, and in the last few months have gotten some cool things done.  For one, they actually got some revenue for the product.  Another thing, Robert Scoble (huge in the blogosphere, in case you don't already know about the guy) heard about it and used it to capture his time at Le Web conference recently.

But none of that really matters to me today; today what's important is that Rich and I are going to snapcast tonight's college football bowl game, the Fiesta Bowl between TCU and Boise State.  Rich is from Texas; he's rooting for TCU.  I'm from the Pacific Northwest, so I'm gonna root for Boise State.

G-Snap! has added a really nice way of incorporating Twitter feeds and individual tweets into a snapcast; we're going to be trying out this feature in tonight's snapcast.  Check it out here; it's free, and you can either just watch the conversation or feel free to join in with the crowd.

Oh, and it's not just for mobile phones; it looks great on your favorite desktop browser.  Rich has made sure of that, no matter what browser you're using.

See you at the game tonight!

Powered by ScribeFire.

Quick post here: I just wrote about a fantastic 15-minute talk I found on the TEDIndia site.  If you want to see a great way to visually present information, watch the whole 15-minute talk.  I was engrossed the whole time, and the speaker is excellent.

Powered by ScribeFire.

Okay, so that title probably needs a little bit of explaining, if you're not following the saga of the OpenSolaris Source Juicer on a daily basis as I am.  I'll get back to the good news about Movable Type 4.31 in a bit; meanwhile, here's the deal with the "finally" part of this entry's title:

Sun's ISV Engineering group has been working for a while to get a pile of open source applications into the OpenSolaris "contrib" repository, which is to say, a place where you can look to find extra stuff that doesn't come with the stock OpenSolaris release.  "contrib" is by no means the only repository of cool extras -- check out the multimedia repo for example -- but it's a good place to put apps in general.  So we do.

We use this thing called the "Source Juicer"; it's a robot that builds software packages from source code, if you give the robot some instructions on where to grab the source from the web, what other software the app depends on, and how to build the app.  Once built, you ask people to test the app and vote on whether it's good-to-go or not.

Every once in a while, the Source Juicer people decide to look at the list of apps that have been favorably voted on, and they promote these apps from a repository called "pending" to the "contrib" repo.  If you look at this list of apps that have been so promoted, what you'll notice is that for the past almost-a-month, the only guy who's gotten his apps promoted are some dude named "Roboporter"; that's not a nickname, that's a clever mechanism the Source Juicer folks use to get a whole big batch of packages into a repo if it looks like the rules for building those apps are really straightforward, or really similar to other apps they've set up for the roboporter.  The roboporter is another 'bot.

Anyway.

We've had Movable Type in the "pending" repo for a while now, and we've had it tested and ready to go for over a month.  (here's proof)  Finally, Eric Reid tells me that the app got promoted into the contrib repo.

Bottom line: if you like the Movable Type blogging engine, you've now got a simple way to create a blog site, protect its data, or to do development and test snapshots of sites you create with Movable Type.  Just install it on OpenSolaris and ZFS will make your data nice and safe.

Check it out, let us know what you think.

Powered by ScribeFire.

So, in the last couple of weeks I've upgraded my home media server.  For example, I have increased the amount of storage and now have two ZFS pools:

1. the root pool (rpool), two 250GB disks in a mirror;
2. a media pool, four 2TB disks in a RAIDZ setup.

ZFS tells me my media pool has about 5.4TB of space to play with; that's plenty for now.  Also, I upgraded to the June 2009 OpenSolaris release (2009.06) and now Time Slider works, so I can easily save incremental backups of my data.  That is a huge relief.

I also moved all of this data (using zfs send | zfs receive) from one computer to another; the prototype computer was using about 240 watts of power in its steady state; the "production" machine uses about 80 watts steady-state.

And speaking of power savings, I just learned of a great blog entry by Nathan Fiedler describing how he built a low-power OpenSolaris NAS box.  He has some great pointers to information about tips on building your own system from scratch if you're new to doing it, and an Ars Technica system buyer's guide.  Plus, his description of his own system (including bill of materials) is nicely written and cleanly laid out.

Anyway, one of the things I wanted to get working was media streaming from the home media server to our home's TiVo Series 3 DVR.  It's working, using the open source streambaby software.  Streambaby was pretty easy to set up and get working, but it's not perfect.  The most common thing I'd like to do with it is to play back TiVo files (*.tivo) that Galleon has backed up for me.  The problem: .tivo files are encrypted with the TiVo Media Access Key (MAK), and streambaby doesn't have a way of decrypting and streaming the resulting MPEG-2 file.  A poor hack to deal with this is to do the following:

1. use Galleon to download the content from the TiVo to the media server
2. use tivodecode to decrypt the downloaded .tivo file, saving the .mpeg file in a separate directory
3. configure streambaby to point to that directory of .mpeg files
4. delete the original .tivo file

That's a waste of time, but it lets streambaby stream the content back to the TiVo.  I would prefer a solution that somehow lets you stream the original .tivo file back to the TiVo, but I don't know if the TiVo knows how to deal with that.  I don't know why it couldn't but when I tried using streambaby to stream .tivo files back to the TiVo whence it came, the TiVo didn't like it.

The good news is that I now have a stable media server in the garage that can archive our TiVo content and play it back to the TiVo.  Now, when the TiVo's disks become corrupted for whatever reason, we haven't lost the programs that we really wanted to keep around.

If only TiVo would find a way to use ZFS as the device's internal filesystem.  I'd never have to worry about losing shows again.

Powered by ScribeFire.

So, I wanted to use the latest build of OpenSolaris to take advantage of some new features.  I am using the June 2009 release (2009.06) and upgrading to the latest build means using a development build.  I was thinking the easiest way for me to switch to the development builds is to go into the Package Manager application, add the publisher "http://pkg.opensolaris.org/dev/" (and give this publisher a name like "development"), tell Package Manager to set this as the preferred publisher, then delete the "opensolaris.org" publisher from the list because I no longer need that one; I'm getting the software I need from the dev publisher.

Well, that's nice in theory but it didn't work for me, and there's a bug for that (here, check it out).  Turns out that for now, you must name your preferred publisher "opensolaris.org".  So the relatively simple workaround is to create a publisher called "opensolaris.org", and have it point to "http://pkg.opensolaris.org/dev/", then delete the original publisher that is called "opensolaris.org".

Worked for me, and now I'm using build 128 (b128) and it's working (mostly) like a charm.  There are other minor things going on with that build, but I've been able to work around them so far with no problems.  I'm happy.

Powered by ScribeFire.

I've been doing some nice upgrades to my home media server: I just put 4 2TB drives in it in a raidz configuration.  This now gives me over 5TB of fault-tolerant storage; if one of those disks fails, my data is still fine and I have time to buy a replacement drive and pop it into the machine so that ZFS can heal itself.

But I ran into a problem when trying to transfer my data from one machine to another.  I would try "zfs send <filesystem> | ssh zfs receive <receivingFilesystem>" of a filesystem that is perhaps 100GB large, but the transfer would never complete.  It seemed to go fine until maybe 10GB of content was sent, then the transfer would stall, and the sending side would complain of a time-out, then quit.  When I logged into the receiving machine (which I earlier wrote about here, so you can see the parts list), I found that it could no longer see the network.

I looked into it; turns out that there's an OpenSolaris bug with the RGE network driver, which is what my computer uses for its on-board Realtek Gigabit Ethernet circuitry.  The community really came through for me here.

Here's a discussion thread that talks about what is going on with the bug, and several versions of a fix that, after several iterations, did the trick for me.  I tried a couple of versions of the developer's fix, and finally one version worked like a charm.  I've used it to successfully transfer over a TB of data.  I just hope this fix makes it into an OpenSolaris build soon; the developer thinks it will take a few months for it to get accepted, which sounds sub-optimal for an open source project that is trying to get support from a community of helpers.  But 3 months is better than nothing, and I'm grateful that somebody created a fix.  Nice job, masa; thank you!

I had been running OpenSolaris 2008.11 on one of my machines for a long time, and when I recently upgraded it to the June 2009 (2009.06) release, all of a sudden the remote desktop control feature stopped working.  I mean, I could type "vncviewer " and would see the desktop initially, but it just didn't refresh its display.

Turns out I'm not the only one who ran into this problem, which doesn't seem to be fixed but at least it has a workaround.  The workaround is to turn off visual effects for desktop actions.  To do that, go to the System menu, choose "Preferences -> Appearance" and you will see a panel that looks like the image below.  Be sure to check "None:" for the visual effects, and vncviewer from other computers (including Macs, which is where I first saw the problem) should go away.

Here's a link to the discussion I found about this problem.

And here's what the preferences panel looks like in OpenSolaris 2009.06 and later builds:

Powered by ScribeFire.

I've talked about g-snap! before; I still think it's one of the most innovative ways of sharing live events with other people.  It beats Twitter hands-down, and is a more natural way of doing what people call "live blogging".  Plus, G-Snap! was designed from the start to work well on mobile devices; the whole idea is that you may be on the move when a favorite event of yours is on, but if you could stay in touch via your mobile device, you can still be a part of the community, part of the action.

The next Snapcast  coming up is tomorrow's (that's Thursday, December 3rd) Pac-10 college football game between Oregon and Oregon State.  The outcome of the game will determine which team wins the Pac-10 conference title and therefore goes to the Rose Bowl.  Both teams are playing great football; the game should be a treat to watch.

So check it out by going here, and join in the trash talking, or just check out how communities interact on G-Snap!

A while ago, I wrote several blog entries about what I did to set up a media server at home.  I'm writing this blog entry to wrap things up with some details about how much it all cost, and the software I'm running on the computer now.

Background

I decided to design a "media server" for home that would be the main data storage for our family's music, photos, recorded TV shows and movies, and personal documents and backups of our home directories on the computers we commonly use at home.  I had a few objectives for the media server:

1. All of this data would be in a single computer that we could grab and stick in the car in case of emergency.  Friends of ours lost their house, but they had the foresight of having all of their personal data (over a terabytes' worth) on a single file server, so when they evacuated the house, they didn't lose any personal data.  Seemed like a great idea to me.
2. The media server would store data reliably; i.e., I assumed that a disk would fail, and I wanted the media server to be able to continue working in case of failure.
3. Reduce noise in the house as much as possible.  That meant putting the server in the garage where it could make as much fan noise as it needed to, but we wouldn't hear a thing in the home office or where we watch TV.
4. Keep the cost down to a reasonable amount.  This was not the primary factor, but it was important enough for me to pay attention to when shopping for the components.

What I Purchased

Here's the build of materials for the computer I put together (I'm sure prices on these will have gone down since I bought them in October 2008).  Total price of the system: $742.  If you take out the cost of the disks, the rest of the computer cost $345.

• Motherboard (newegg.com, $62.99) : ECS Elitegroup A780VM-M2 Micro ATX AMD Motherboard.  Supported the cheap AMD processor I wanted, and had plenty of on-board SATA ports (6).  Remember, I want all the disk to be in a single enclosure for easy, snatch-and-grab transportation.
• Memory (newegg.com, $69.99): GSkill 4GB (2 x 2GB) DDR2 SDRAM (PC2 6400).  No reason to get greedy, and it keeps power draw and price down for the system.
• CPU (newegg.com, $59.50): AMD Athlon 64 X2 5000 Brisbane 2.6GHz Socket AM2 65W Dual-Core Processor.  Nice price for a processor that does everything I need in a media server that can also run some apps.  More on the apps in a bit.
• Disks (newegg.com, $396.80): 2 Seagate Barracuda 1.5TB 7200 RPM SATA internal disks.  Prices have gone down considerably for the storage, but this was a fair enough price last year.
• Case (Fry's Electronics, $130): Antec Sonata III Mid Tower ATX Case.  Got good reviews, came with a 500W power supply which is plenty for what I was putting in it, and has plenty of ports and internal drive bays for the storage.

How I Set Up The Media Server

Here is the first blog entry I wrote that describes my thought process about using ZFS on OpenSolaris to store our home's data.

Here is the second blog entry I wrote, giving all the details about what I needed to do to format the disks and set up the ZFS pools and filesystems.  This was based on the OpenSolaris 2008.11 release; I plan to upgrade to a more recent release (probably an OpenSolaris build after the June 2009 release; I'd love to try the new ZFS deduplication feature).  Everything has been running fine, with the exception of the TimeSlider feature for doing automated ZFS snapshots.  That was fixed in the June 2009 release.

What I'm Running On The Media Server

The media server is doing a few things for us at home; here are the services it's providing:

• iTunes music storage.  We have a Mac mini that we use for iTunes; all of our music is in iTunes, and the Mac mini NFS-mounts a filesystem from the media server.  I tried both CIFS (Samba) and NFS.  I preferred NFS, but it seemed to have troubles with the Mac as an NFS client.  After reading James Gosling's blog entry on what he observed with Macs and Solaris and NFS, I made a tweak on the Mac mini and everything has been working smoothly ever since.  We make this library shareable to our other Macs on the home network.
  
• iPhoto storage.  The same Mac mini also stores all of our photos.
• TiVo media backup and playback.  There is a great open source Java application called Galleon that uses the TiVo Home Media Engine (HME) API; the API lets you write your own Java apps that show up as part of the TiVo's on-screen menu system, and it lets you talk to the TiVo to grab shows off the TiVo's disk and put shows on there, among other functions.  Primarily, we use it as a way to keep backups of shows that we don't want to lose in case the TiVo's disk crashes and loses data.  This has happened several times, mostly due to unforeseen loss of power.
• Personal finances (via Intuit's Quicken software).  To do this, I run the free, open source VirtualBox software.  I used to use VMware, but VMware isn't free and it doesn't run on any version of Solaris.  VirtualBox runs on every operating system I use, it's free, and it's improving much faster than VMware is.  So, I run Windows XP as a guest OS inside VirtualBox, and use that guest OS to run Quicken.  When I'm feeling like the guest OS is getting slow or am worried that it's collected viruses, I just blow away that guest image and go back to an earlier snapshot.
  

My Wish List

Are any of you doing something similar with your home setups?  If so, maybe you have done some of the things I'd like to set up but haven't gotten done yet.  Here's my wish list of apps and features I'd like to add, and I'd like to do it all natively in OpenSolaris (in other words, none of these services would need to run in a guest OS like Windows or Linux under VirtualBox):

• DLNA server running natively in OpenSolaris.  DLNA will give me the ability to share and stream my media (photos, music, video) to the PlayStation 3.  I've played with several solutions; more on this in a moment.
  
• Live streaming of TiVo content from the media server back through the TiVo.  Galleon lets me transfer shows to and from the TiVo, but not play them live.  This is a feature that the ReplayTV product had (I still miss that product, and am sorry they went out of business), and their Java app was superior in several ways to Galleon.  But, it looks like somebody has written software to let you play TiVo shows from your media server straight through the TiVo.  I just haven't gotten it running on the media server yet.
  

Powered by ScribeFire.

Sometimes my TiVo gets confused after a power outage, and it takes forever to reboot (I think it's trying to recover its files which takes a while when you have over 1TB of storage connected to the TiVo).  Sometimes, I lose some of my shows, and that makes me sad.

So I decided to find some software I could get running on OpenSolaris so I could use my home media server as a backup device for my TiVo shows.

I did it, using the Galleon open source Java application; here's how.

Now, if somebody smarter than me would just port the TiVo software to OpenSolaris, we'd have the benefits of ZFS on a TiVo box.  That would be fantastic.

Powered by ScribeFire.

I just published a short how-to video showing how you can easily install Acquia Drupal on OpenSolaris.


Sun's ISV Engineering group has been working with Drupal for a couple of years now; I also use Acquia Drupal to develop and maintain this web site, which also hosts my personal blog.  The Drupal community is amazingly vibrant and growing by leaps and bounds, and Acquia has done a nice job of commercializing the Drupal open source content management software package.  Acquia Drupal is still free, but it has some nice additions that any Drupal user or developer would want.

Anyway, check out the video; it's about 8 minutes long and walks you through the steps to find the catalog of third party applications, then get Acquia Drupal and configure it on your OpenSolaris system.  It's easy, and it just works.

Oh and by the way: props to our Sun Learning Exchange web site and staff for making it so darned easy to publish content.  Nice job, folks!

Powered by ScribeFire.

Movable Type is another open source blogging platform, written in Perl, and is one of several I've been playing with over the past few weeks.  Sun's ISV Engineering group has been working on getting several of the leading open source content management systems and blogging engines into OpenSolaris via the neato installation system, to make it as easy as possible for fans of these blog engines and CMS's to get working right away, with little hassle.

Eric Reid in ISV Engineering recently made Movable Type available in the OpenSolaris "pending" repository of software, the "pending" meaning "please somebody test this package so we can vote it into the "contrib" third-party software repository.  So, I spent some time checking out the installation and configuration process for Movable Type.  Here is what I did and what I observed.

First, I set up the OpenSolaris to find packages from the Source Juicer "pending" repository, with these two steps:

1. type "pfexec pkg set-publisher -O http://jucr.opensolaris.org/pending jucr-pending"
2. type "pfexec pkg refresh"

Next, I launched the Package Manager application and chose the "jucr-pending" repository from the pop-up menu on the right side of that application's user interface.  After Package Manager thought about its new catalog of apps for a moment, I saw a list of hundreds of packages available to me.  I used the search field to type "type", and found the "movabletype" package.  Selected that, saw that it lists several other packages as dependencies (i.e., those other packages had better be installed for Movable Type to work correctly), then I installed it.  Nicely, the Package Manager installed MovableType and its dependent packages for me.

One more thing: we've configured MovableType to use the PostgreSQL database by default; before quitting Package Manager, I selected the main "opensolaris.org" repository, then selected PostgreSQL 8.3 packages (the server and client packages, plus one for docs and manual pages) and installed those.

Next step: get the database running.  I don't know much about PostgreSQL (shame on me), so I cheated by looking at this nice blog entry to show me what I needed to do to configure it as a nice Solaris service, so that it will launch for me automatically.

Got the database running, created a database called "mtype", granted all privileges to that database for a user I created, and was ready for the next step: web-based configuration of MovableType.

I launched my web browser and went here: http://localhost/movabletype to bring up the easy-to-use web-based installer.  It checked the system, told me everything was fine (except that I'm missing a few optional components that won't hurt me; I tested on a minimally-installed system), then asked me some questions about the name of the database to use, which user and password, then it took a minute to churn.  No problem, it was ready to go.

After install, MovableType brought up a web page asking me to login.  I did so, created my first blog entry (which was incredibly easy to do), then saw it published.

I made a minor mistake when I set up MovableType: it asked me where I wanted to publish my blog.  I mistakenly said "http://localhost/my_first_blog".  I should have said "http://localhost/movabletype/my_first_blog", because the web server has write access to that directory.  This messed up subsequent preview and blog navigation actions, but when I entered the correct URL, everything worked just fine.  Pilot error, not the package's problem.

Next, I created a new user, gave that user access to the original blog (so now I have a blog that can have two different authors), and tested it by posting a guest blog entry.  Yep, it worked.  Next, tried posting a comment to the blog.  That worked, too.

I futzed around a little bit with the Movable Type features.  Although I didn't spend a ton of time on it, everything seemed to work as expected.

So, check it out, Movable Type users.  Does Movable Type worke okay for you on OpenSolaris?  It was pain-free for me, but let us know what you think.

Powered by ScribeFire.