Lubos's Weblog Lubos Kosco's Weblog

I am happy to trackback to Tronds post on the release:

http://blogs.sun.com/trond/entry/opengrok_0_8

I hope to get the docs (http://hub.opensolaris.org/bin/view/Project+opengrok/installdescription) up to date soon, so people can start enjoying opengrok even more !

The other notable thing to mention is, that we're switching to 3 month release train, which means that the wait is over ... one can get the goodies of new opengrok release every 3 months !

Hello all

we're proudly announcing the release candidate 1 for opengrok 0.8, get it here:

http://hub.opensolaris.org/bin/download/Project+opengrok/files/opengrok%2D0.8%2Drc1.tar.gz
or in SVR4 form
http://hub.opensolaris.org/bin/download/Project+opengrok/files/OSOLopengrok%2D0.8%2Drc1.pkg

Rough draft of new features:
- javadb history cache (gives dir history, file listing per changesets, etc.)
- multiple project search
- upgraded lucene to 2.4.1
- improved OpenGrok script (2 commands should be enough to get your sources indexed and running in tomcat/glassfish : ./OpenGrok deploy ; ./OpenGrok index )
- auto generated opensearch (after a search you get a generated header with appropriate projects selected, so you can easily add it to your search providers)
- several UI improvements (line # toggle, line # is a link now, hover over revision in annotate shows message, show/hide files touched by changeset, doubleclicking/Enter on project takes you to its xref)
...

Note , that you SHOULD reindex from scratch, for javadb historycache setup read http://src.opensolaris.org/source/xref/opengrok/trunk/README.txt#129

More bugs fixed here(just the ones tagged with closed in 0.8, there were definitely more):
http://defect.opensolaris.org/bz/buglist.cgi?query_format=advanced&bug_status=RESOLVED&bug_status=VERIFIED&bug_status=CLOSED&product=opengrok&target_milestone=0.8

Please report your bugs to opengrok-discuss@opensolaris.org, or preferably using http://defect.opensolaris.org

Thanks in advance for your help / patches / feedback
Trond, Knuth, Lubos, Jorgen, Jan, ... ( https://www.ohloh.net/p/opengrok/contributors )

P.S. more info about commits and some statistics data about project: https://www.ohloh.net/p/opengrok/commits

Couple of weeks ago we (CZOSUG) were invited to do a presentation of OpenSolaris at Technical University in Kosice, Slovakia(tiny, but beautiful country in the heart of Europe) by a local university computer network laboratory.

singing fountain before the theatre in Košice

I traveled there with my colleague Vita and we presented OpenSolaris, Virtualbox, showcasing some Solaris technologies and deep diving into ZFS, Zones and Dtrace.

I have a picture from the event, so see for yourself :

me showing the intro to OpenSolaris

During the presentation I promised to answer questions, which I took offline, on my blog, so here you go guys (ENG is in round brackets) :

Q: Funguje "copy & paste" medzi virtuálnymi strojmi(VM) vo VirtualBoxe?
(Does copy & paste work between two virtual machines(VM) in VirtualBox?)
A: áno, proste funguje, AK su správne nainštalované a funkčné "Hosťovské doplnky", je nutné podotknúť, že pre väčšinu systémov copy and paste medzi VM a hostiteľom funguje len v grafickom rozhrani (X, alebo Windows)
(Yes, it just works, IF the "Guest Additions" are correctly installed and working, we have to mention, that for majority of OSes copy and paste between VMs and host works only in graphics (X, or Windows).)

Q: Dá sa pripojiť virtuálny disk .vdi k systému na čítanie aj bez naštartovania vo VirtualBoxe?
(Can one mount a virtual disk .vdi to a system for reading without having it shared in running VirtualBox?)
A: áno, no nie je to triviálne a zatiaľ oficiálne podporované, odporúčam blog kolegu Renauda a určite si pozrite forum virtualboxu pre pripájanie vdi obrazov (myslím si, že táto aplikácia bude adoptovaná virtualboxom, prípadne väčšinou distribúcii a umožní transparentné pripájanie obrazov diskov vo formáte .vdi)
(Yes, but it is not trivial and not officially supported, I recommend a blog from colleague Renaud and be sure to check virtualbox forum project for mounting vdi images (I think that this application will be adopted by VirtualBox and maybe by majority of distributions and will allow transparent mounting of disk images in .vdi format))

Q: Ak odinštalujem balíček v IPS pomocou pkg, budu odinštalované aj závislosti, ktoré si balíček doinštaloval a už nie sú viac potrebné?
(If you uninstall a package using IPS pkg command, will also its dependencies, that it installed, be uninstalled , when they are no longer used by any other packages?)
A: zatiaľ nie. Pokiaľ viem, je to funkcionalita, ktorú má v dnešnej dobe iba Debian a jeho deriváty v podobe deborphan, ktorý bol prednedávnom pripojený do apt (obdoba pkg, no pre správu .deb balíčkov). Začal som ale mailové vlákno na danú tému, tak uvidíme aká bude odozva vývojárov IPS (skontrolujte dané mailové vlákno pre viac informácii).
(Not yet. As far as I know, it's functionality, which is today only Debian and its derivatives in the form deborphan, which was recently annexed into apt (it's something like the pkg, but for managing .deb packages). I started a mail thread on the subject, so we'll see what will be the response of IPS developers (please check the mail thread for more info).)

Q: Dajú sa do virtuálneho stroja vkladať lokálne disky?
(Can you use local hard drive in virtualbox?)
A: nie priamo virtualboxom, no jedna z možností čo tento problém rieši nepriamo je iSCSI - viac v príkaze VBoxManage, príp. návod z blogov Sun-u(hostiteľ je Solaris a jeho iscsi server) alebo v príručke virtualboxu
(Not directly by VirtualBox, but one of the options that addresses this issue indirectly is iSCSI - more in help of VBoxManage command, eventually check instructions from sun blog(host is Solaris and its iscsi server) or in the manual of virtualbox

Q: Ako je to správou veľkého počtu virtuálnych strojov vo Virtualboxe?
(How to manage a large number of virtual machines in VirtualBox?)
A: momentálne si človek musi vyrobiť vlastné riešenie, založené buď na príkazovej riadke (pomocou VBoxManage v pdf príručke, sekcia VBoxManage reference) alebo na web services - vboxwebsrv (viac vo VirtualBox SDK). Je istý predpoklad, že Sun doplní podporu pre virtualbox aj do xVM Ops Center-a, čo je hlavný nástroj na správu serverov momentálne vyvíjaný v Sune.
(Currently, one must create his own solution, based either on the command line (using VBoxManage pdf manual section VBoxManage reference) or on web services - vboxwebsrv (more in the VirtualBox SDK). It could be that Sun will add support for VirtualBox in the xVM Ops Center, which is the main management tool for servers currently being developed by Sun.)

I left the tricky question till the end:
Q: Aká je výkonová strata virtualizácie pri porovnaní Xen-u a Virtualbox-u?
(What is the virtualization performance loss when comparing Xen and VirtualBox?)
A: táto otázka je strašne zaujímavá, no bola vzťahovaná na porovnanie VirtualBox-u a Xen-u. A to je sčasti ako porovnávať hrušky a jablká. VirtualBox plne emuluje virtuálny počítač, no Xen musí mať príslušné jadro OS "portované na platformu" Xen(i86pv). Takáto paravirtualizácia v Xen-e je blízka výkonu jadra priamo na HW (uvádza sa, že záťaž spôsobená virtualizáciou je menej ako 10%), no cena je práve uvedená modifikácia OS. Sun xVM Server založený na technológii Xen(od v.3.0 vyššie) ponúka ešte možnosť tkzv. HVM módu - čiže plnej virtualizácie a tá by sa už možno dala porovnávať s virtualboxom. Tu sa ale dostávame k ďalšiemu problému a tým je testovacie kritérium. Potom na konci vysvitne, že pri istých podmienkach je vhodnejší VirtualBox a pri iných zase Xen, zaujímavý článok ohľadne tohto som našiel na fedoraproject.org - je to ale iba porovnanie na Linuxe, takže zase nie stopercentne objektívny článok a v podstate aj dosť starý. Ďalšia debata by mohla nasledovať, ak by sme porovnávali virtuálne stroje bez a s hosťovskými doplnkami. Takže z mojej strany by som asi navrhol, nech si každý skúsi a rozhodne sám. Objektívnu odpoveď si netrúfam vysloviť.
(This question is very interesting, it was asked regarding VirtualBox and Xen. This is unfortunately the same as comparing apples and pears. VirtualBox fully emulates a virtual computer, but Xen must have the OS kernel "ported into the platform" Xen (i86pv). Such paravirtualization in Xen is close to running on bare metal hardware (it says that the overhead caused by virtualization is less than 10%), but the price for it is the porting effort. Sun xVM Server based on the Xen technology (v.3.0 from above) has a so called HVM mode - full virtualization, and could be compared with VirtualBox. Here, however, get to the next problem and this is benchmarking. Then in the end one figures out, that in certain situations VirtualBox is better and in other Xen, an interesting article on this is on fedoraproject.org - it's but only a comparison on Linux, so it's not that objective and in fact, quite old. Further discussion could follow, if we compare the virtual and host machines without guest additions. So from my side, I would suggest you go & try each and decide for yourself. I don't dare to answer this objectively.)

So I'd like to thank YOU for reading this blog and guys in Slovakia for coming to our presentation.

Two days ago I found out that students from that university started Slovak OpenSolaris User Group - SKOSUG after our presentation, so I'd really like the outcome of the whole trip and who knows, maybe I get a chance to come back and present more

Oh ... and find the slides on the CZOSUG archive site and I hope the video will be uploaded to the videoserver of the university lab soon.

So after some time passed since my tests of OpenSolaris on USB key/disk install I decided to try again.
This time however I only had a crappy Akasa Integral USB enclosure with some old 40GB disk (this device popped up as "Initio MK8025GAS USB Device" ) to be used as installation target.

You might ask why I call it crappy ... simply ... if you plug this disk into systems that don't care that much about standards it will work. However in Solaris it wasn't detected by rmformat, just dmesg confirmed its presence, but that was it.
With Milos Muziks help and Chris Gerhards neat scsi.d we figured out that the problematic one is not the usb driver(note that builds before 103 were unusable with it ), but scsa2usb driver which one might know as mass storage driver (or: SCSI to USB bridge driver , if I refer to man scsa2usb) which was showing that the device doesn't respond correctly.

With above troubleshooting done it was fairly easy to nudge Solaris to talk to such disk, just add this line at the end of

/kernel/drv/scsa2usb.conf :


attribute-override-list = "vid=0x13fd pid=0x550 rev=0x312 reduced-cmd-support=true";


which will basically enable restricted SCSI commands for the device and then the usb disk will pop up in rmformat, or in format -e
Note that I got the vid, pid and rev from appropriate section of prtconf -Vp output.

Now the question was, how to get this "workaround" into some installer and install OpenSolaris on such disk.
First thing was to get a medium, that can be somehow modified, e.g. an USB installation image such as the one from genunix.org. Then I used an usb install stick creator from Hiroshi Chonan to get a bootable stick done ... but wait ... the stick didn't see my "crappy" usb disk!

Sooo ... I started my virtualbox instance of OpenSolaris, I plugged in the USB, told virtualbox to mount it to the virtual machine.
Once I mounted the USB disk, I found and backed up boot/amd64/x86.microroot from the media. Then I needed to edit the scsa2usb.conf inside this microroot.
First rename to .gz , then to gunzip , then lofiadm -a of the unzipped file, then a mount , then edit the file, add the proper line, then umount, then lofiadm -d , then gzip it and then move it back to USB flash as boot/amd64/x86.microroot. Then unmount the usb and you're ready to boot on "real iron". Note that I edited amd64 bit one, since amd64 bit kernel started for my installation (I am on Intel 64bit , but not Itanium), one would need to edit the boot/x86.microroot for 32 bit.

After this change on USB installer (see, if I was using CD installer this step would be a bit more harder ), you can easily install from USB key onto the crappy USB disk, which is now magically available and installer does it job (hooraayyy!).
After installation is done, you just reboot and enjoy - but keep the disk in the same USB port (otherwise you'll hit the bug with zfs boot and physical path which I think in current state just panics the kernel )!

I usually run my lab and dev OpenSolaris, Solaris, Debian, RHEL ... machines inside vmware(a server solution, mainly because I run it on my work server on stable debian for production reasons). For a long time I was a happy user of VMware Server 1.x , then I decided to move to version 2.0

Guess what ... every change brings consequences ...

After migration of VMs, upgrading to latest VM HW version I booted my debian and opensolaris dev boxes & wanted to login.

All my machines networking is setup by NAT & DHCP from VMware.

Hmm ... at that time I felt victim of very interesting problem. The login took ages (cca 5mins in comparison to 2s in previous version).

SSH does a reverse resolving of hostname when you try to connect to the target & this did a call to naming services.

This took so long I even got a fatal error complaining about ssh monitor not responding in OpenSolaris.

The same behaviour happened in Debian, so this pointed out a problem with VMware.

After doing some "snoop" and "truss -Df sshd -ddd -p 222" I figured out that the DNS packets are taking a long time and all those packets ask for a "localdomain" host.

localdomain ? WTF ?

Then the root causing and fixing was rather fast & the solution was in the file /etc/vmware/vmnet8/dhcpd/dhcpd.conf .

This file provided two options regarding naming:

option domain-name-servers 192.168.1.2;

option domain-name "localdomain";

Simply commenting out second option (so option domain-name) got me rid of the ridiculous resolving problem.

Funny thing is that I always thought "localdomain" is ignored by dns client, but as usual assumptions are evil

I don't think something like that would happened to me if I used virtualbox, but who knows, thanks god we have snoop and truss

For those of you, who don't know logwatch, it's a perl script, which prepares log reports from your system and sends them to you.
On Linux (read: Debian ) systems this small tool is enabled by default & I enjoy it, because it can help monitor your machines and give you an overview what's going on there.

Sooo the other day I was thinking about putting up logwatch for tomcat on one of our Sparc boxes with Solaris 9 (yeah, I know, S10 rulez, but we need S9 for legacy stuff).
But since I am not only a man of thinking, I went & downloaded logwatch source, put it on my box, installed and was wondering if logwatch supports tomcat out of box ... bummer! It's not!

Hmm ... now I was thinking ... tomcat logs which I wanted to be analyzed are not the usual catalina.out, but I turned on access logging, which formats out strings very similar to apache(http).
So why not go & use logwatchs "http" service ???

YEAH, small change to <logwatch_basedir>/default.conf/logfiles/http.conf to add:
LogFile = localhost_access_log*
LogFile = logs/localhost_access_log*
and it all worked like charm

But since I needed one more feature - counting of unique IPs which accessed my server, I had to develop it and that's how MY tomcat service saw the light of world ... it supports apache log files as well, so to avoid double reports when installing this service, use service="-http" .

Now I am a happy user of weekly report from my tomcat logwatch service (download in Resources section, NO Support)

Remeber:
logwatch --help is your friend
--logdir is a good parameter as well

Resources(download):
tomcat_logwatch.tgz
(you will probably need to unpack it to /usr/share/logwatch/ )

Phew ... my second blog in one day

Well ... finally some info and news about N1, which after some struggles and internal reorgs comes out with a new general name - System Management software (not official yet, no hate/black mails pls ).

Albeit it's not opendatacenter.org you can still get some info and publish/submit your knowledge, tips and tricks on BigAdmin Information hub http://www.sun.com/bigadmin/hubs/sysmgmt/


That page contains already lots of know-how and plenty of links to discover. So to learn new ways of automating your common administration tasks, spend some time checking those pages out.

N1SM can help out with OS install automation, patch distribution, firmware upgrades and monitoring.

N1SPS can wrap up your service provisioning processes, giving you easy control on what goes where, how it will be set up and easy manage. I saw actually an implementation of N1SPS for easy to use and manage deployment of web application, which saved that customer lots of troubles and money doing it the old way. Hmm, not mentioning that you can see the whole history of actions in N1SPS, so it's very easy to audit.

Slowly but surely projects with code are being unveiled. Let's see if repositories will be filled and people start to submit / check out the code. First swallows(as planned) are SPS plugins on http://n1-sps-plugins.sunsource.net/ . What might follow is the core of N1SPS and then maybe N1SM and SunMC.

Well, can't wait to see people hacking plugins into SPS .

Soo

in the first week of October I had the honor to attend CEC 2006 conference as one of the GENO (Global Engineering Organization) guys. Hmm, Dan Berg has mentioned GENO in his blog as well, and he has there a "backdoor" picture with me & ferrari laptop on my lap sitting just in front of big Greg P .

Anyways, there have been a lot of musings and news about CEC, especially from our hosts Jim Baty, Hal Stern and Dan Berg, so I am not going to duplicate them.

Overall one could have seen a lot of great technology and innovation we are making here at Sun, but since I am a N1(Systems management) guy, I've mostly had a look on those pressos & demos.

However I really enjoyed one session - Bryan Cantrills presso on new network provider in dtrace, be sure to check it out here.

All in all San Francisco was very nice & it was my first time to Pacific coast so I had those "big eyes". Driving around the coast and through SF was a nice experience as well, hehe, I got pretty addicted to "cruise mode" in those US cars ... wondering if I will remember how to drive with manual transmision . During the weekend I drove with my coleague Milos to Santa Cruz, Monterey, saw San Francisco Red Bull fly day and visited Muir Woods.

And yes, I had no flowers in my hair (next time, I'll wear them ) ...

Some time ago, I was in need of this functionality.
This little silly script, however very uneffective will check which apps listents on the given port number.
Maybe I should think about writing something more effective ... or just file a Request For Enhancement against netstat on Solaris?

Script get_port_pid.sh:

#!/bin/bash

for i in `ls /proc`
do
pfiles $i | grep AF_INET | grep $1
if [ $? -eq 0 ]
then
echo Is owned by pid $i
fi
done


Prepared for a long time the openn1.org (or www.opendatacenter.org) will be public in close time span.

What's this (N1) all about?

N1SM - System Manager - provision OS(RH,Solaris,Windows), patches, firmware, monitor server HW, provision N1 Grid Engine, diskless stations

N1SPS - Service Provisioning System - provision services (like Weblogic, Websphere, Oracle, ...) and ev. OS, Zones, ... write your own plugin for what you need

N1GE - Grid Engine - job distribution(dispatching) and management software across grids (of different kinds)

So to be able to manage your datacenter better, easier and save $$$, these tidbits of software are ready to serve.

Be sure to check the site out then, at least for the fancy flash videos and demos.

Edited 6.5.2005 - Sorry for a false alarm folks, the site still needs some more ice cream and fudge to suit your taste
But hey, stay tuned!

Some thoughts and unanswered questions about pluggable(modular) architecture of todays software.

Ah, nice to be blogging again :]

Today however, I got a complain reason ... plugins or modules (how do U like to call 'em?) ...

Been updating Firefox lately and naturally to get it to the Opera like level I had to update dozen of plugins. One of them broke my browser in that kinda way, that the session manager plugin works odd now and I keep getting a stupid preferences alert window whenever I open a new firefox window (not tab).

Yeah, I am an engineer, so I am going to find what the prob is and fix it, but it's just a waste of my(our) time caused by lack of testing before release.

Why do I even care to write about it? Because living on the edge just brings such stuff, but still I think, most of the probs could be omited.
A similar thing happened to me with our Netbeans, when I installed a bunch of plugins(modules) and one of them broke my Tools/Options dialogue (it crashed down).

I mean, it's really nice to be able to put your own functionality to the product, but couldn't be the integration improved a little bit?

If I should compare Opera against Firefox or JBuilder against Netbeans(Eclipse) I'd say, that thing is the only difference(functionality is almost the same, user experience lacks the integration feeling). One can say it's the same reg. Windows and Solaris (Linux).
Hmm, maybe the "distros" idea isn't that bad. I really like NB5.5 enterprise preview or other our product built upon Netbeans. But than again it's a problem with confusion - all the names for the same thing ... hmm, guess that's the tax for high expectations from things (yeah, converging to a very political topic).

Wondering what can be done. Better testing before including to a public download/update site? Hire some integration and testing teams? Or just leave it on people - enthusiasts who create their own "distros" (leave the community to solve it)?

I'll definitely update this article after the talk with Roumen on this reg. to Netbeans

Till then just have a good one ...

Update:

Well, living on the edge ... test by yourself, or just wait and get a stable software ... sigh

Btw. I had to reinstall Firefox and redownload all modules, before that I cleared all registers and leftovers. Sometimes to start over from scratch just clears the way for better things

Today I have one from those yet another bookmark'o'article & I would like to ask you all for a favour.

First I thought ... Code reviews? ... isn't this just another bugger and slowdown process?
But ... honestly, it's not. I even like it. Looks like the most of the people don't
But trust me, it's really usefull & maybe it could teach you something new. At least you get in touch with other developers .

There's one article I read recently:
http://www.developer.com/java/other/article.php/3579756

I haven't read anything about Code Inspections, but in general they seem to me more like a teaching process, but hey! Two times do the measurement & then cut

Finally, I am happy, that there are people researching different ways of how to make code development a better and more superior process.

& before I press "Post to weblog" ... I would like to ask you all to put a comment with a link to a good web page or give a good advice (you use, or have experienced) about Code reviews or Code Inspections (so we can all get a better approach to them )

Thanks!

Some people drink, the other take drugs, some just do sports, then there are people who pray or worship (whatever they care about) ... everyone just does something that helps him to relax (yes, whatever you call it )

My stuff is definitely music.

Not that I like only one kind of music ... I can listen almost to everything.
But some genres just make me feel good.

I have a very fine collection of classical music. The most inspiring is Beethoven, there's no dispute about it
I could name a few others that I like, e.g. Debussy and his Claire de lune, Gorecki and wonderful Symphony No 3, Saite Gymnopedie, Massenet Meditation, and you probably know Offenbach and his Barcarolle (at least from this: http://www.imdb.com/title/tt0118799/).
The funny thing is, that I like to play it too, but not often I have so much time to learn the whole piece, so you might see me play at least part of the score (if it wasn't replaced in the mean time by some bug fixing ideas)

Another stuff is good chillout. If you want make a good gift, give me chillout (music)
My favourite ones are from CDM (www.cafedelmarmusic.com , gosh they rebuilt the site, those backgrounds are awesome!) and I will visit Ibiza and this club for sure (visit CDM, see the sunset and then I can die ),
but lately I started to enjoy somehow asian influenced Budha Bar.

Ha ... I also have an album, which is very unknown to me(but is a pleasure to hear). It was a gift from my sis, unfortunately only in form of digital music without any info
So ... if anyone knows something about "chill out - schhh" album, let me know!

And naturally combination of those two is an overkill So maybe next I'll write something about my chillout classic albums collection