Sunday March 30, 2008

How to install the TwonkyMedia UPnP server on Solaris

In my last post, I compiled an installed the MediaTomb UPnP server on Solaris in order to stream movies, photos and music to my PS3 and it worked well. But I wasn't quite satisfied with it's features: No support for tags/covers in AAC encoded music (>95% of my music library is encoded in the superior AAC format) and a few other quirks here and there. So I decided to try the TwonkyVision TwonkyMedia server.

Unfortunately, the guys at TwonkyMedia (now PacketVideo) don't support their TwonkyVision server on Solaris (yet?). Only Linux, Windows and MacOS X are supported. The absence of answers to a Solaris request post in their forum isn't very encouraging. TwonkyMedia is closed source and only commercially available (EUR 29.95) which means you can't even compile it yourself on Solaris. At least there's a trial period of 30 days. Does this mean no ZFS and other Solaris goodness to TwonkyMedia?

Fear not, this is exactly what Branded Zones in Solaris 10/OpenSolaris are all about! They allow you to install a  Linux distribution inside a Solaris 10 Container. The BrandZ framework then seamlessly translates Linux systemcalls into Solaris systemcalls. The result: All the goodness of Solaris, such as ZFS, FMA, DTrace and whatnot, even for closed source or otherwise problematic Linux applications. So, here's how to run the TwonkyMedia server on a Solaris x64/x86 machine (sorry, no SPARC, different CPU architecture):

• Set up a standard lx branded Zone. Here's a short and sweet tutorial on how to do it. In my case, I used ZFS for the zone root path. This gives me compression and the ability to snapshot the Linux root filesystem whenever I like.
• I used the CentOS tarball from the BrandZ download area to install a standard CentOS zone. Quick, easy, free, works well for most cases.
• After having installed the CentOS Linux branded Zone and before the first boot, it is a good idea to make a ZFS snapshot of the root filesystem, just in case. You can later use the snapshot to revert the zone to it's freshly installed state or to easily clone more zones like this in the future.
• After the first boot of the Linux zone with zoneadm -z zonename boot, you can login to it's virtual console using zlogin -z zonename. Now, setup basic networking from  inside the Linux zone by editing the /etc/sysconfig/network file. Then, you can login through ssh -X into the Linux zone and run graphical configuration tools such as redhat-config-network to configure DNS, set up users, etc.
• Now, download the TwonkyMedia server from the Linux zone by using wget http://www.twonkyvision.com/Download/4.4/twonkymedia-i386-glibc-2.2.5.zip and follow the TwonkyMedia installation guide.
• You should now have the TwonkyMedia server up and running from within a Linux branded zone on Solaris! Connect to it through your webbrowser at http://your.servers.ip.address/:9000 and configure it's various settings to your taste.

This is it, actually it's much easier than compiling MediaTomb, but it comes at the cost of having to pay after the trial period, if you like it. Above, you see a picture of TwonkyMedia, running in an lx branded zone on Solaris, streaming AAC music from my favorite Chilean band "La Ley" to a PS3. Notice the cover art and song info to the bottom left that is not available with MediaTomb today for AAC encoded music.

I'm now going to write to the TwonkyVision support department at support@twonkyvision.de and ask for a real Solaris version. After all, if they expect their customers to pay for software, they ahould at least provide a real binary. If you're interested in getting TwonkyMedia to run natively on Solaris too, join me and send emails of your own to them or post to their forums.

"How to install the TwonkyMedia UPnP server on Solaris" has been brought to you by Constantin's Blooog.
This entry was created on 2008-03-30 11:44:58.0 PST and is associated with the following tags:
brandz container dlna linux opensolaris ps3 solaris streaming twonky twonkymedia twonkyvision upnp zones
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.
Comments [4]

Thursday March 20, 2008

How to compile/run MediaTomb on Solaris for PS3 and other streaming clients

Before visiting CeBIT, I went to see my friend Ingo who works at the Clausthal University's computing center (where I grew up, IT-wise). This is a nice pre-CeBIT tradition we keep over the years when we get to watch movies in Ingo's home cinema and play computer games all day for a weekend or so :).

To my surprise, Ingo got himself a new PlayStation 3 (40GB). The new version is a lot cheaper (EUR 370 or so), less noisy (new chip process, no PS2 compatibility), and since HD-DVD is now officially dead, it's arguably the best value for money in Blu-Ray players right now (regular firmware upgrades, good picture quality, digital audio and enough horsepower for smooth Java BD content). All very rational and objective arguments to justify buying a new game console :).

The PS3 is not just a Blu-Ray player, it is also a game console (I recommend "Ratchett&Clank: Tools of Destruction" and the immensely cute "LocoRoco: Cocoreccho!", which is a steal at only EUR 3) and can act as a media renderer for DLNA compliant media servers: Watch videos, photos and listen to music in HD on the PS 3 from your home server.

After checking out a number of DLNA server software packages, it seemed to me that MediaTomb is the most advanced open source one (TwonkyVision seems to be nicer, but sorry, it isn't open source...). So here is a step-by-step guide on how to compile and run it in a Solaris machine.

Basic assumptions

This guide assumes that you're using a recent version of Solaris. This should be at least Solaris 10 (it's free!), a current Solaris Express Developer Edition (it's free too, but more advanced) is recommended. My home server runs Solaris Express build 62, I'm waiting for a production-ready build of Project Indiana to upgrade to.

I'm also assuming that you are familiar with basic compilation and installation of open source products.

Whenever I compile and install a new software package from scratch, I use /opt/local as my base directory. Others may want to use /usr/local or some other directory (perhaps in their $HOME). Just make sure you use the right path in the --prefix=/your/favourite/install/path part of the ./configure command.

I'm also trying to be a good citizen and use the Sun Studio Compiler here where I can. It generally produces much faster code on both SPARC and x86 architectures vs. the ubiquitous gcc, so give it a try! Alas, sometimes the code was really stubborn and it wouldn't let me use Sun Studio so I had to use gcc. This was the path of least resistance, but with some tinkering, everything can be made to compile on Sun Studio. You can also try gcc4ss which combines a gcc frontend with the Sun Studio backend to get the best of both worlds.

Now, let's get started!

MediaTomb Prerequisites

Before compiling/installing the actual MediaTomb application, we need to install a few prerequisite packages. Don't worry, most of them are already present in Solaris, and the rest can be easily installed as pre-built binaries or easily compiled on your own. Check out the MediaTomb requirements documentation. Here is what MediaTomb wants:

• sqlite3, libiconv and curl are available on BlastWave. BlastWave is a software repository for Solaris packages that has almost everything you need in terms of pre-built open source packages (but not MediaTomb...). Setting up BlastWave on your system is easy, just follow their guide. After that, installing the three packages above is as easy as:
  

  # /opt/csw/bin/pkg-get -i sqlite3
  # /opt/csw/bin/pkg-get -i libiconv
  # /opt/csw/bin/pkg-get -i curl
  

• MediaTomb uses a library called libmagic to identify file types. It took a little research until I found out that it is part of the file package that is shipped as part of many Linux distributions. Here I'm using file-4.23.tar.gz, which seems to be a reasonably new version. Fortunately, this is easy to compile and install:
  

  $ wget ftp://ftp.astron.com/pub/file/file-4.23.tar.gz
  $ gzip -dc  file-4.23.tar.gz | tar xvf -$ cd file-4.23
  $ CC=/opt/SUNWspro/bin/cc ./configure --prefix=/opt/local
  $ gmake
  $ su
  # PATH=$PATH:/usr/ccs/bin:/usr/sfw/bin; export PATH; gmake install

  Notice that the last step is performed as root for installation purposes while compilation should generally be performed as a regular user.
  

• For tag extraction of MP3 files, MediaTomb uses taglib:
  

  $ wget http://developer.kde.org/~wheeler/files/src/taglib-1.5.tar.gz
  $ cd taglib-1.5
  $ CC=/usr/sfw/bin/gcc CXX=/usr/sfw/bin/g++ ./configure --prefix=/opt/local
  $ gmake
  $ su
  # PATH=$PATH:/usr/ccs/bin:/usr/sfw/bin; export PATH; gmake install
  

• MediaTomb also uses SpiderMonkey, which is the Mozilla JavaScript Engine. Initially, I had some fear about having to compile all that Mozilla code from scratch, but then it dawned on me that we can just use the JavaScript libraries that are part of the Solaris Firefox standard installation, even the headers are there as well!
  

 Compiling and installing MediaTomb

Now that we have all prerequisites, we can move on to downloading, compiling and installing the MediaTomb package:

• Download the MediaTomb source from http://downloads.sourceforge.net/mediatomb/mediatomb-0.11.0.tar.gz
• Somehow, the mediatomb developers want to enforce some funny LD_PRELOAD games which is uneccesary (at least on recent Solaris versions...). So let's throw that part of the code out: Edit src/main.cc and comment lines 128-141 out by adding /* before line 128 and */ at the end of line 141.
  
• Now we can configure the source to our needs. This is where all the prerequisite packages from above are configured in:
  

  ./configure
  --prefix=/opt/local --enable-iconv-lib --with-iconv-h=/opt/csw/include
  --with-iconv-libs=/opt/csw/lib --enable-libjs
  --with-js-h=/usr/include/firefox/js --with-js-libs=/usr/lib/firefox
  --enable-libmagic --with-magic-h=/opt/local/include
  --with-magic-libs=/opt/local/lib --with-sqlite3-h=/opt/csw/include
  --with-sqlite3-libs=/opt/csw/lib
  --with-taglib-cfg=/opt/local/bin/taglib-config
  --with-curl-cfg=/opt/csw/bin/curl-config
  

  Check out the MediaTomb compile docs for details. One hurdle here was to use an extra iconv library because the MediaTomb source didn't work with the gcc built-in iconv library. Also, there were some issues with the Sun Studio compiler, so I admit I was lazy and just used gcc instead. 

• After these preparations, compiling and installing should work as expected:
  

  gmake
  PATH=$PATH:/usr/ccs/bin:/usr/sfw/bin; export PATH; gmake install
  

Configuring MediaTomb

Ok, now we have successfully compiled and installed MediaTomb, but we're not done yet. The next step is to create a configuration file that works well. An initial config will be created automatically during the very first startup of MediaTomb. Since we compiled in some libraries from different places, we either need to set LD_LIBRARY_PATH during startup (i.e. in a wrapper script) or update the linker's path using crle(1).

In my case, I went for the first option. So, starting MediaTomb works like this:

LD_LIBRARY_PATH=/opt/csw/lib:/opt/local/lib:/usr/lib/firefox
/opt/local/bin/mediatomb --interface bge0 --port 49194 --daemon
--pidfile /tmp/mediatomb.pid
--logfile=/tmp/mediatomb.log

Of course you should substitute your own interface. The port number is completely arbitrary, it should just be above 49152. Read the command line option docs to learn how they work.

You can now connect to MediaTomb's web interface and try out some stuff, but the important thing here is that we now have a basic config file in $HOME/.mediatomb/config.xml to work with. The MediaTomb config file docs should help you with this.

Here is what I added to my own config and why:

• Set up an account for the web user interface with your own user id and password. It's not the most secure server, but better than nothing. Use something like this in the <ui> section:
  

  <accounts enabled="no" session-timeout="30">
    <account user="me" password="secret"/>
  </accounts>
  

• Uncomment the <protocolInfo> tag because according to the docs, this is needed for better PS3 compatibility.
• I saw a number of iconv errors, so I added the following to the config file in the import section. Apparently, MediaTomb can better handle exotic characters in file names (very common with music files) with the following tag:
  

  <filesystem-charset>ISO-8859-1</filesystem-charset> 

• The libmagic library won't find its magic information because it's now in a nonstandard place. But we can add it with the following tag, again in the import section:
  

  <magic-file>/opt/local/share/file/magic</magic-file>

• A few mime types should be added for completeness:
  

  <map from="mpg" to="video/mpeg"/>
  <map from="JPG" to="image/jpeg"/>
  <map from="m4a" to="audio/mpeg"/>

  Actually, it should "just work" through libmagic, but it didn't for me, so adding theses mime types was the easiest option. It also improves performance through saving libmagic calls. Most digital cameras use the uppercase "JPG" extension and MediaTomb seems to be case-sensitive so adding the uppercase variant was necessary. It's also apparent that MediaTomb doesn't have much support for AAC (.m4a) even though it is the official successor to MP3 (more than 95% of my music is in AAC format, so this is quite annoying).

• You can now either add <directory> tags to the <autoscan> tags for your media data in the config file, or add them through the web interface.

This is it. The pictures show MediaTomb running in my basement and showing some photos through the PS3 on the TV set. I hope that you can now work from here and find a configuration that works well for you. Check out the MediaTomb scripting guide for some powerful ways to create virtual directory structures of your media files.

MediaTomb is ok to help you show movies and pictures and the occasional song on the PS3 but it's not perfect yet. It lacks support for AAC (tags, cover art, etc.) and it could use some extra scripts for more comfortable browsing structures. But that's the point of open source: Now we can start adding more features to MediaTomb ourselves and bring it a few steps closer to usefulness.

"How to compile/run MediaTomb on Solaris for PS3 and other streaming clients" has been brought to you by Constantin's Blooog.
This entry was created on 2008-03-20 15:07:08.0 PST and is associated with the following tags:
compiling diy howto installing media mediatomb open playstation ps3 software solaris source streaming
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.
Comments [3]

Monday March 17, 2008

CeBIT 2008 impressions

CeBIT 2008, the largest IT trade show worldwide, is over. This must be my 9th CeBIT as a Sunnie, boy does time fly fast. Here are a few impressions from my point of view.

Thanks to Detlef, who set up an Ultra 40 M2 with a current Solaris Express and Sun xVM Server for us (here's a nice writeup (sorry, in german) on how he did it, in case you want to try out xVM yourself), buildup was done really quickly. We had two monitors attached to the machine and thanks to NVIDIA's "nvidia-settings" tool that they ship with the Solaris NVIDIA drivers, setting up Twinview was a piece of cake too.

Then we set up the Compiz window manager to run on our Solaris Ultra 40 M2. Few people know what it is (it adds some 3D eye candy to your desktop, similar to Apple's) and even fewer know that it runs on Solaris as well. Thanks to Erwann, installing Compiz is just a matter of running a script. Even if you have an ATI card, you're likely to be able to run Compiz, thanks to Minskey's preliminary driver. It runs just fine on my Acer Ferrari 4000 laptop!

But then we found out that running many virtual OSes on a machine requires quite some amount of memory. Our 8 GB inside the Ultra 40 M2 wasn't enough for the different versions of Solaris, Linux and Windows that we had installed. So we hunted down an unsuspecting little Sun Blade X6220 module and ripped it open for an extra 4 GB. To the right, you see Ulrich performing the upgrade, Systemhero-like (i.e. no anti-static mats or straps, those are for sissies...). Now there was enough air to breathe for our virtualized OSes, the booth was ready to go!

Day 1 wasn't the busiest day, as expected, but it kept us quite entertained. Mario Heide from the german POFACS podcast stopped by and we explored a few things we could do for future episodes.

High-End Visualization: There was also quite an interest from the automotive industry in trying the Sun Fire X4600 M2 8-socket Opteron Server with up to 256 GB of RAM with the NVIDIA Quadro Plex  VCS external graphics cards as a really big workstation, or a network visualization server. The LRZ supercomputer center near Munich is already using such as setup to provide virtualized remote graphics power to their researchers and now the manufacturing industry is starting to like the idea. An ideal companion for this is Sun's suite of visualization software that provides both scalable and shared approaches to high-end visualization. Try it out, it's free and open source.

Optimizing AMP: Another popular question was: "How can I optimize the AMP stack on Solaris and Sun Hardware?" Each day, I pointed about a dozen customers to our Cool Stack homepage, which is part of the Cool Tools developed by Sun for the UltraSPARC T1/T2 processors. The Cool Stack is simply a set of popular web apps (you know, Apache, MySQL, Perl, PHP, Tomcat and friends) which have been precompiled by Sun for Solaris on both x86 and SPARC architectures. Since we compile with Sun Studio compilers using the right options and integrate them with selected Solaris technologies, such as the cryptographic framework, using the Cool Stack is both easy to do and it provides great out-of-the-box performance.

All the other days were very busy. Loads of people, loads of questions lots of interest in Sun technologies, both in hardware and in software. The great thing about this particular CeBIT and the new Sun booth, now in Hall 2 was that the people who came by were all relevant to Sun. We hardly had any "bag-rats" at all, so I guess this is as good as it gets in terms of visitor quality. Visitors ranged from high-level IT executives through middle-management, system administrators, hackers, students and Sun/Solaris enthusiasts.

Sun Ray and Sun Secure Global Desktop: We also had schools looking at our Sun Ray and Sun Secure Global Desktop solutions as a flexible, secure, cost-effective and eco-friendly infrastructure for their schools. Actually, Sun Ray technologies were among the hottest topics discussed during this CeBIT at the Sun booth, not just for schools but also for any kind of environment that is sick and tired of having to upgrade Windows or Linux PCs every couple of years. Also call centers, branch offices and a couple of special applications such as kiosks are very good fits for Sun Rays.

Sun xVM was another hot topic. Having been at the Sun xVM pod with Ulrich and Detlef, we explained numerous times how the Sun xVM Server adds value to the work of the Xen community by providing Solaris technologies as the better foundation for virtual machines of all OSes. The Solaris Fault Manager can monitor your hardware and trigger virtual machine migration before the hardware starts failing for real, increasing uptime for your virtualized applications. This can work hand in hand with the Solaris Cluster, which adds high-availability features to virtualized OSes. ZFS is a great tool for providing fast, flexible, integrity-checked and powerful storage through iSCSI, NFS, CIFS or other protocols to virtualized environments. And there's much more, for example the Solaris Crossbow project which adds fully virtualized and bandwidth-managed network devices to the picture, enabling full network-in-a-box virtualization approaches. Oh, and when a virtual machine fails, you can debug it with DTrace, too. Levon has some nice examples about DTrace and Xen working together!

Sexy Hardware: No Sun booth at CeBIT without showing off some tin and this year was no exception. For starters, we had a datacenter with Sun's newest UltraSPARC T2, AMD and Intel based servers, both in rack-mount and in blade form factors. Of course we also had some storage arrays and a big tape library to show off.
But the big eyecatcher was the Sun Modular Datacenter S20 (formerly known as "Project Black Box") which was so big and so eye-catching that we had to place it outside the halls, near the Intel pavillion. Our heroic product manager Ingo explained everything about project Black Box to customers, including more than a handful of TV stations. Even at 4 o'clock in the morning, for the ARD TV station's breakfast TV show...

Back to Solaris: The nice thing about Solaris at CeBIT 2008 was that we hardly needed to explain to people that it is free and open source. Most visitors already knew this and came to visit us specifically to learn some more about a particular Solaris feature, grab a Solaris Express Developer Edition DVD or ask questions about how to best deploy Solaris in their environment. One system administrator actually thanked us for producing our CSI:Munich ZFS video because it helped him gain his boss' support for deploying ZFS in their company. The boss just said: "If this really works, then we need to roll it out now!" (Of course it "really worked"). Actually, ZFS was one of the most popular discussion topics, and I logged in to my home machine more than once to show some real life, production snapshots, pools and other ZFS features on a living, breathing system.

Getting Started with Solaris: We handed out a lot of Solaris Express: Developer Edition DVDs and to get people going and avoid the initial humps of first-time Solaris users, we pointed visitors to the same essential and useful links over and over again. This inspired me to post an entry into the german Solarium blog with the 7 Most Useful Solaris and OpenSolaris links. Now I only need to point customers to a single website for all their initial Solaris needs: The Solarium.

Helping and Learning: But we learned a lot of new stuff, too. Not only are Ulrich and Detlef great sources of endless Solaris knowledge (them being OS Ambassadors at Sun), I also had a number of very illuminating conversations with customers and visitors. Thorsten Ludewig of the Wolfenbüttel University of Applied Sciences updated me on the state of the art of digital picture frames. A guy from Konstanz University pointed me to a small company in Switzerland called "PC Engines" that manufactures small form factor systems with good quality. I'm looking for a small, low-power system as a backup server at home and this might be it. He's running NetBSD on these systems for small and home server tasks, but I wonder if they work with Solaris as well. At only 256 MB it might be a stretch but not impossible. Other options I'm considering are VIA's Artigo kit or maybe a standard Via motherboard in an ITX case after all? Let me know if you have experience with Solaris on very small, very low-power machines.

Meeting Customers and Interests: CeBIT, like any major trade show is a great way to connect with customers and interests. Sometimes it's a way of meeting people you only knew virtually. In this case, we had three fans of the Systemhelden.com podcast HELDENFunk visit us at the booth: Graefin, Chaosblog and Unruheherd. All three came in white Sun T-Shirts which could only be rewarded with new black Systemhelden.com T-Shirts :). We had a great time during the Sun booth party that day and according to Chaosblog's latest entry, they seem to have had a fun time at CeBIT a well.

In closing, this was probably one of the best CeBITs I've ever had. Customers and partners like Sun, they are excited about our technology and they want more. Some know us because of our Software and were suprised to learn that we have hardware, too (this is a good sign), some come to see our hardware and discover our software portfolio (this case is slightly more common) and all want us to win, which is a good feeling :).

Check out my CeBIT 2008 photo gallery on SmugMug for some more impressions of the Sun booth @ CeBIT with comments.

Oh, Rolf brought some beer to celebrate. Cheers!

"CeBIT 2008 impressions" has been brought to you by Constantin's Blooog.
This entry was created on 2008-03-17 08:35:17.0 PST and is associated with the following tags:
2008 cebit hannover opensolaris solaris sun tradeshows xvm
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.
Comments [1]

Friday March 14, 2008

Presentation on Web 2.0 to the German Unix User's Group (GUUG)

Yesterday I was invited to present on Web 2.0 during the German Unix User's Group's (GUUG) annual conference called the "Frühjahrsfachgespräch" (Spring Topic Conversations). The day before, Ulrich and I did a ZFS workshop during the same conference.

I was originally planning to show a video before the presentation, but I discovered too late that I forgot to install mplayer on my laptop. It's actually as easy as "/opt/csw/bin/pkg-get -i mplayer" (which I'm doing right now), if you have Blastwave installed. Here are two great videos to show during such occasions.  

About 35 people came and we had some interesting discussions after the talk. Some people like "The new web" because of its new possibilities of participation. Some are scared by fear of privacy, profiling and spam issues. My personal opinion is that the best way to deal with it is to participate, learn and adapt one's lifestyle to the Web 2.0 reality. If you don't like what Google comes up with when you search for your name, then update your online profiles on the different social networks, start a blog (or update it more often) and make sure that the good stuff you do shows up on the web somewhere. I only blog about 2-3 times a month, but this is enough for Google to put my blog on the top three links when searching for my name.

Here are the german slides to my current web 2.0 presentation, and an english version is available too. The slides are meant to complement the speaker, not to substitute him or her, so they may only be of value to people who attended the session. If you want to get the presenter as well and you are in Germany, tell your Sun sales rep to do a Web 2.0 workshop with your company :).

Yes, Alec, I know I should record myself on video... I will.

"Presentation on Web 2.0 to the German Unix User's Group (GUUG)" has been brought to you by Constantin's Blooog.
This entry was created on 2008-03-14 03:29:12.0 PST and is associated with the following tags:
ffg german guug presentation web2.0
You're welcome to use this Permalink , add a comment below or send your feedback to constantin at sun dot com.