Dana's Weblog

It's been months since I gave everyone an update on the Second Life viewer on Solaris. So here it is...

First, you need to know that the viewer is now available outside Sun. The pre-built packages are now available off of amazonaws.com, see Jeff Barr's blog entry for the details. I want to thank these folks for putting the effort in and making the viewer available to everyone.

If you want to build the viewer yourself, please refer to my past posts on building the libraries and tweaking SConstruct. As for the patches themselves, I've submitted them under Second Life "jira" issues and, as such, I ask that you download the patches from there. The issue numbers are:

• VWR-4800 - Use Solaris kstats when running on Solaris
• VWR-4802 - add definitions for stricmp and strnicmp
• VWR-4804 - Add necessary includes to llsys.h
• VWR-4805 - add LL_SOLARIS to #if in lltimer.h
• VWR-4806- llvfs changes
• VWR-4807 - add Solaris to #if in llglstubs.h
• VWR-4808 - featuretable_solaris.txt patch
• VWR-4809 - llfloaterreporter.cpp
• VWR-4810 - add Solaris required changes to llappviewer_linux.cpp
• VWR-4811 - add LL_SOLARIS to #if in llfilepicker.cpp
• VWR-4812 - include llstring.h in llfloateranimpreview.cpp
• VWR-4813 - include llstring.h to llfloaterimagepreview.cpp
• VWR-4814 - include llstring.h in llfloaternamedesc.cpp
• VWR-4815 - add Solaris specific platform to llfloaterreporter.cpp
• VWR-4816- include Solaris in llstartup.cpp
• VWR-4817 - don't include gMovieMaker on Solaris builds
• VWR-4818- add LL_SOLARIS to #if in llfilepicker.cpp
• VWR-4819 - add LL_SOLARIS to linux #if in moviemaker.h
• VWR-4820 - add solaris_tools to newview

If you want the latest audio updates, I suggest you refer to VWR-2662 for the latest patches. Be sure to read the comments included in VWR-2662 so that you get everything you need. The patches are added in the order documented in the jira issue, the result is a viewer with OpenAL supported audio as well as GStreamer supported audio streaming. However, streamed audio only works on OpenSolaris because the GStreamer support in SL requires GStreamer 0.10 (delivered in OpenSolaris) and Solaris 10 currently supports GStreamer 0.8.

Seriously, save yourself a bunch of time, and just take the pre-built packages mentioned above. Attempting a SL build can induce sleeplessness, outbursts of frustration, and possible tears.

My plans for the future work on the Second Life viewer depends priorities here at Sun. I have plenty of other work on my to-do list so we'll see. But if I do get the thumbs up, I plan on doing the following:

• Reduce the number of libraries included in the SL package in favor of those bundled with OpenSolaris under the Project Indiana initiative.
• Add patches to the cmake branch of the Second Life build tree. yes, that's right, SCons is no longer going to be used to build SL. I'm not sure I like cmake but it does provide a good cross-platform (Windows, OS X, Linux, and Solaris) build environment.
• See what, if anything, can be done to get the Linux version of the Vivox client working under Solaris. I'm thinking that perhaps Project Indiana may help here.

Cheers!

This entry covers the steps required to build version 1.18.3.5 of the Second Life client on a Solaris 10 x86 and OpenSolaris x86 system.[Read More]

Well folks, it has been a fun couple of months. After getting the SecondLife viewer ported to Solaris x86 I thought the move to SPARC would be a breeze. NOT!!! It turns out there were quite a few CPU alignment assumptions made throughout the code that caused headaches. But before I get into that I need to put in a plug for gccfss (gcc for SPARC System) which is a great tool to use if you have a huge application like the SecondLife viewer, have a community of folks who share development duties and therefore tie you to gcc, but want superior performance on SPARC. gccfss is the gcc 4.0.x front end on top of the Sun Studio 12 backend. You therefore can compile code that assumes gcc-isms while still able to use the SPARC performance tools. I also need to plug Sun Studio 12 which has many gcc compatible features. If you own the application in question I highly advise you to invest the time in getting your code to compile in Sun Studio 12. There are far too many reasons to describe here so look at the SunStudio overview page for more details. In either case, gccfss and SunStudio 12 are free so you'd be a fool if you didn't invest the time and give them a try. SunStudio is available for Solaris and most Linuxes.

OK, now back to SecondLife. After deciding to go with gccfss, I still had data alignment to deal with. Alexey Starovoytov came to the rescue advising me to add asm("ta\t6"); to SL's main(). This little trick enables a corrective trap handler on SPARC that will fix unaligned references. For those who need a deeper definition, John Harres provided the following:

Since this is a user land trap, it ends up in the user side of the trap table, which effectively adds 0x100 to the trap number. This ends up in:

GOTO(.fix_alignment);           /* 106  do unaligned references */

in the trap table. It ends up setting p_fixalignment in the proc structure, which then is noticed during trap() if you're taking a T_SYS_RTT_ALIGN trap from user land. WIth that flag set this results in the call to do_unaligned(). Therefore, each time the program takes an alignment trap, do_unaligned() is called. One would expect the overhead to be rather minimal unless one is doing a lot of unaligned references.

If you need a source reference, Alexy pointed me to simulator.c as well as a discussion of trap 6 in the Sun Studio Performance Library doc.

I can't thank Alexy and John enough for their help.

Once I got passed the alignment problems I then ran up against Open GL Library issues on SPARC. The SL deveopers assume the GLext, originally called the GL Extensions for Linux Library, exists. Well currently on SPARC GLext does not and at the moment I've #ifdef'ed around these calls to get a clean compile and wound up making the viewer ugly. This is what I'll be attacking next.

Here's a few other things that were done to the Viewer for Solaris compatibility:

• All the libraries required by the viewer but not bundled with Solaris were built for the SPARC using the same methods described for the x86 port. They were then copied into the linden/libraries/sparc-sunos5 directory for the SCONS build.
• The Lindens have dedicated pages of source to figuring out the type of x86 processor is in use. Instead of using their assembler calls and because said assembler does not apply to SPARC I used getisax(2) to extract the cpu info. The cpu kstats were used to determine the processor brand and clock rate while on x86/x64, getisax(2) was used to further define which CPU extensions were in use.
• The in window color depth was reduced from 32 to 24 bits on SPARC to allow compatibility with more of the frame buffers available on SPARC.
• The Viewer assumes the X.org X-Window System server. It currently references the X.org server logs to determine how much memory is configured on the GPU. A true kludge. On SPARC X.org is not yet supported so this code is useless and it's ignored. According to its documentation the SDL library provides such information but when one inspects the SDL source one finds the variable but discovers it's not ever set. Doh! Therefore for now, the viewer assumes the maximum amount of memory is available, the default action when it can't be determined by any other means.
• One of the largest stumbling blocks was big endian conversions on SPARC. I spent weeks trying to band-aid my way through fixes. Then last week, as he was adding the Solaris changes to the master repository at Linden Labs, Soft Linden discovered that a header file was missing from one of their message passing sources - a LL bug that I missed. That header was the one that defined LL_BIG_ENDIAN. Once I added that include everything fell together and I proceeded to rip out my endian changes. Without Soft's help I would have wasted a ton of more time "fixing things". Thanks Soft!
• The SL community has offered an openal alternative to the FMOD audio library. Using the patches offered for openal I was able to add audio support to Solaris. There is a rumor that there may be an FMOD available for Solaris in the near future so that option may be viable soon as well.

So where does the Solaris port stand? Except for movie support the viewer on x86 works fine. The SPARC port still needs work to address the GL compatibility issues. Like I said earlier, I'll be attacking those next.

Lastly, I got approval from Sun Legal to sign the Linden Labs Contributor Agreement and am now officially able to give LL copies of the Solaris patches. I was asked not to distribute any 3rd party libraries outside Sun so I'm hoping someone else will build a version for general distribution. Soft Linden has been working with me to merge those changes into the official SVN repository so they should be available to everyone soon.

If you are interested in building the SL viewer on your own, you should start by reviewing the pages at the SecondLife Open Source Portal.

Back in the beginning of April I was tasked to get a Second Life (SL) client running under Solaris. Luckily, Siva had done some preliminary work so I thought all I had to do was pull the source and start a compile, right? NOT! Second Life is not a trivial build. One problem I ran up against is that Open Second Life does not stay up to date with the source available from Linden Labs(LL) and earlier this month LL decided they need to ensure clients were current so they imposed a new serial number check during authentication. Hence, I was forced to pull a copy of the latest client source from the SecondLife wiki page and merge in the changes required to support Solaris. Since then, I also added a bunch of my own changes to better support Solaris.

Before I get into the discussion of how the build went, I should point out that Linden Labs has a detailed discussion of how to build their client on Linux at their Wiki site that one should read carefully before even attempting to do their own SL client build.

Let's first discuss the build environment. The Second Life source is supplied in two pieces: the source itself and the art work. One needs to download both and then inflate them on top of each other. Next, the build does not use make. It uses SCons which is a Python-based script that does essentially what make does. Instead of a Makefile one has to maintain a SConstruct file. It's a fairly easy environment to get used to and like makefiles, one has to be VERY careful about indentation. In the Second Life client, the SConstruct file is in linden/indra.

As discussed in the Wiki build notes, There are several third party libraries required to build the client. I built these using --prefix=/opt/opensl so that they all got placed in a directory that I could control and so that I knew these libraries would not conflict with anything in /usr/local or the other "freeware" delivered with Solaris in either /usr, /usr/sfw, or /opt/sfw . Here's the notes I took while working with each library:

• curl - you need 7.16.x (SFWcurl from the Companion CD delivers 7.10.3)
• boost - a C++ library, it's huge
  configure --with-python=/usr/sfw/bin/python --prefix=/opt/opensl
• Apache2 (for apr, aprutil) - I just built Apache2 from the source and copied the libraries I needed.
• xmlrpc-epi 0.51
  configure --prefix=/opt/opensl
  You also need to use gmake.
  IMPORTANT:xmlrpc-epi uses an internal data structure called "queue". Unfortunately that structure conflicts with the Solaris STREAM queue so I changed xmlrpc-epi's queue to XMLRPCqueue. You won't notice the conflict until you start compiling the SL client.
• libjpeg - I used the one bundled with Solaris, see /usr/lib/libjpeg*
• Vorbis 1.1.2 & Ogg 1.1.3, (SFWvorb is version 1.0 which is too old but SFWoggl looked fine.)
  configure --with-ogg=/opt/opensl --prefix=/opt/opensl
• ElfIO 1.0.3
  ./configure --prefix=/opt/opensl
  The make does not build a shared library and LL recommend doing so (I didn't.)
• openjpeg 1.1.1
  ./configure --prefix=/opt/opensl
  LL wiki page recommended patches for openjpeg but the latest source includes these changes.
• expat 2.0.0
• curl 7.16.1
  configure -- with-ssl=/usr/sfw --prefix=/opt/opensl
• SDL 1.2.11 (SFWsdl = 1.2.5 which is too old.)
• bison 2.3 (SUNWbison is too old.)
  This one I built and installed in /usr/local/bin and I made sure the SConstruct file used this version.

Once I got everything built, I copied the necessary includes and libs into linden/libraries/"platform" where platform is either i686-sunos5 or sparc-sunos5. (I need to note that I've only just started a SPARC build. More on it in a future blog.)

the SCons command line I used was:

TEMP_BUILD_DIR="/tmp/danaf/secondlife-build" scons DISTCC=no BTARGET=client \
    BUILD=releasefordownload

When ready to create a "releasable" a copy, the SConstruct uses a packaging script called package_client.sh inside a tools directory under linden/indra/newview. Of course in my case this meant having to create a solaris_tools directory and the files/scripts associated with it. I used the Linux equivalents as a reference and quickly decided that it would be better to create a packaging script that created a Solaris pkg instead of the Linux tarball approach. I also decided to hide the client executable in a separate exec directory so that all people will find in the bin are things that can run as is.

Some folks at Linden Labs have said that they'd be happy to work my changes back into their source tree so I'll be sending them "patches" in a day or so.

Lastly, and as proof, here's a couple images off my Ultra-40. The first is the good ol' SecondLife login screen and the second was taken from within the Sun Pavillion.

Second Life Login Screen

Inside the Sun Pavilion @ Second Life