Roman Shaposhnik's

Long time ago in the galaxy not so far away when you had to upload a file to your friend you would call him up and have a ZMODEM session. With this new fad called Internet it is all about SSH of course. But is the default configuration secure enough? Is creating an account for a friend who needs to upload a file to you all you need? How do you communicate credentials? [Read More]

With Intel and AMD introducing Quad core systems and Sun pushing the envelope even further with its first hexadeca-core CPU named Rock the question that software industry has to ask itself is: are we ready to make all these threads of execution do useful work for us? My strong opinion is that we are not ready. A new paradigm is yet to be invented and we don't really know what it should look like. We do, however, seem to know what it should NOT look like. It definitely should not look like POSIX threads. What's wrong with POSIX threads you ask?[Read More]

I would like to invite all of the developers out there to join a bunch of my colleagues and myself in the Live Chat event that we're putting together to discuss anything that is related to the software development on Solaris. We would be answering your questions on Solaris Express Developer Edition, Sun Studio, HPC and talking about all things related to the development process starting from how come we still have /usr/sfw/bin all the way to whether or not iPod support is part of the proper developer's desktop. The event will run from 24th of July all the way to 27th of July every day from 9am-11am PST. All you have to do is just click on the Live Chat link and you should be good to go. The whole thing looks strikingly similar to a "public" IM and is quite simple to use.

P.S. Come on! Bring your embarrassing questions with you, 'cause you know what -- quoting snippets of this upcoming live chat might just be the best way to convince PHBs around here. After all you are the customers and you know what that means.

I have recently come across the video recording of Linus Torvalds himself arguing a case for Git as the ultimate source code management system. The presentation is done in the usual trademark Linus style (which I happen to like immensely since it seems to be the only way to wake up the audience sufficiently enough to be engaged in a conversation) but the issues he tackles are very poignant and go far beyond the merits of a particular SCM system into a realm of what is the best tool for automating highly decentralized peer-reviewed software development model:

1. Distributed vs. centralized development.
   Bottom line: centralized SCMs run against the proverbial Bazaar.
2. Using networks of trust as the key means of conquering complexity of the modern software projects and dealing with compartmentalization of key developers/experts.
   Bottom line: SCM should reflect how humans are wired.
3. How to keep the pace of development activity at the highest possible level (by making it cheap and easy to experiment and not care about breaking other stuff) yet allow the easiest transition path for the changes that seem to be beneficial for the project.
   Bottom line: branch early, branch often.
4. Developers vs. gate keepers and why the problems they face are fundamentally different.
   Bottom line: even the best branching is useless without merging

I find his arguments about why CVS is the most braindead SCM ever invented and why Subversion simply denies itself the right to exist by proclaiming that they are the CVS done right quite convincing. Especially so when they are coming from a guy who has a project of ~22000 files to maintain and does about 25 merges per day. His main pain points with CVS/Subversion hit very close to home:

1. There is no data coherency model to write home about. Which might not be a problem for most projects but certainly is for something as security sensitive as an OS kernel. Basically with CVS (and I think even Subversion) the only way you know that your data got corrupted is when its too late.
2. Branching is waaay too painful because of things like global namespace for branches (read: constant collisions and things like test_12345 branches), all branches being pushed down every developer's throat, etc.
3. Subversion patches certain things as far as branches go, but fails miserably as far as merging is concerned.
4. The tools you use are supposed to make you more productive. Period.

Now, I must admit that as much as he was successful at positioning Git to look as the best tool for the job I am still not convinced on two accounts:

1. Does git really offer a nice way of structuring complex projects like KDE and such? His suggestion of Super projects (workspaced with pointers to individual Git repositories) might be an interesting one, but it certainly requires some practice and experience to be evaluated properly. [2008 Update: It seems that this is no longer an issue. Git got the infrastructure for supporting Submodules and from what it documented on their TWiki looks like they got it right]
2. Is the approach Git takes with making it easier to work with projects as a whole at an expense of treating projects as collection of files the right one? See the problem is -- I'm much more of a lone developer than a gatekeeper. So Git might be optimizing for the role I rarely find myself in (and Linus finds all the time)

But regardless of these concerns I highly recommend you watch the presentation yourself -- it is well worth it. Just keep in mind one thing: just before the BitKeeper (something Linus seems to have very fond memories of) Lary used to develop this little project called TeamWare here at Sun. What was TeamWare? Well it was "a distributed source code revision control system... which BitKeeper seems to share a number of design concepts with".

But people try to apply XML to everything they see nevertheless. Sometimes, however, the results are not just sad, but downright spooky:

Date: Wed, 23 May 2007 10:11:57 +0200
From: Lluis Batlle 
Subject: Re: [9fans] XML
To: Fans of the OS Plan 9 from Bell Labs <9fans@cse.psu.edu>

Lately I've been told at work to use a library in C. Most calls have
the signature

  ErrorType function(const char *xml);

I have to pass to them xmls of more than two levels deep, attributes,
and around ten elements.

When I asked why such interface to a library, claiming that it was
uncomfortable to me, the lib developer told me that in fact
xml-parameter-passing was one of the techniques he liked most, and
helped him solve a lot of problems easily.

One of the great tenets of C++ has always been: "you don't pay for what you don't use". Bjorn seems to use this very line everytime he bashes Java or LISP-like languages. The later is especially noteworthy since he is usually pretty honest in acknowledging that functional languages do posses a number of important qualities such as having an expressive power of writing qsort as a trivial two liner

 qsort []     = []
 qsort (x:xs) = qsort (filter (< x) xs) ++ [x] ++ qsort (filter (>= x) xs)

at the expense of wasting quite a lot of precious resources such as RAM and CPU cycles.

Now, from time to time I see folks showing examples of C++ code bordering on the same expressive power. Take this little word counter ($ wc -w) for example:

int main() {
   std::cout << std::distance(std::istream_iterator(std::cin),
                              std::istream_iterator());
   return 0;
}

It looks impressive if nothing else, and since it is, after all, C++ everybody expects it to perform quite well. But does it?

To answer this question without dragging the reader into the dark realms of assembly language or black art of performance measurements I would really love to have a good old Cfront around. Or any other tool for that matter that would be able to retrace what exactly all the templates and overloaded functions got expanded into. Alas, I don't know of any such tool (if you do -- please leave a comment!). So bear with me while I'll be using my stop watch ;-)

For the speed trial lets compare it to the similar code written in C (and to make it fair I am going to even use scanf instead of a handcrafted code):

int main()
{
    int count;
    char buf[65535];
    for (count = 0; scanf("%s", buf) != EOF; count++)
         ;
    return printf("count %d\n", count);
}

Not that I am surprised, but C++ version ended up being 1.5 slower on my machine. And if you compile the above example into .s file and look at what main() turned out to be you can see a reason why. There is about 6 function calls there. Pretty much nothing got inlined or computed in-place.

Bad implementation of a fine idea? Perhaps (I tried two: G++ and Sun Studio). But it makes one wonder why in 28 years the world hasn't yet seen a good implementation. It's not that the industry hasn't tried, you know.

Am I hearing the ghostly murmur of Algol 68 or is it just my imagination?

For quite sometime now I've been giving these presentations on how the layman's understanding of the Moore's Law is dead and how the world is embracing the multicore (CMT) model. The usual question I get whenever I convince people that this is, in fact, the case seems to be: "But how can I jumpstart my foray into this magical world of parallel computing?" My usual answer to that has been: get your hands on the only 8 core CPU available today in the market -- UltraSPARC T1 by applying for the Try'n'Buy program.

Well, now I've got an even better answer for all of the OpenSource developers out there: you can ssh into one of these things: ssh.sundemo.ru!

That's right. Apply for the account on the Sun Studio Demo Server and hack away!

P.S. Even thought the form is in Russian, here's what the individual fields mean:

1. First name
2. Last name
3. E-Mail
4. Address
5. Phone

The last one is a pretty generic EULA check box.

Ever since Sun Studio compilers and tools started to support Linux we've been present at various Linux shows and LinuxTAG in Germany has always ranked pretty high on the list. This year, however, we had a very special reason to make it to LinuxTAG: it just so happened that the release date for Sun Studio 12 (our first release ever to officially support Linux at the same level Solaris has always been supported) coincided perfectly with the LinuxTAG 2007 in Berlin. Add to it the fact that Solaris Express Developer Edition 6/07 was supposed to have its release around the same timeframe and it doesn't take a genius to figure out that we had to do something extraordinary for all those hardworking C, C++ and Fortran developers out there. The real developers. We had a couple of ideas like having a contest similar to the Google's Summer of Code or just bringing a couple of kegs of beer on site but they all either were too heavyweight to pull off in just a couple of weeks or downright illegal. Finally we've asked ourselves a very basic question: since we all are, in fact developers, what would we, personally, get excited about? I hate to say it but it turned out that the BSD community had beaten us to a punch with the idea that was unanimously deemed to be the most exciting one, so we had to opt out for programmable robots. After all, it is much easier to control a robot, you know.

And that's how controlling a LEGO Mindstorm NXT robot using Solaris Express Developer Edition and Sun Studio has become THE whole point of the contest:


Here's a funny bit of trivia...[Read More]

I'm a fundamental believer in scratching your own itch to be the best kind of motivation for software development. And Project D-Light is not an exception here: if I want it to be useful for others I have to make it useful to myself. Of course, given that Project D-Light tries to utilize the most cutting edge features of the DTrace technology it just makes sense to run it on the most recent build of Solaris. And the fact that we are building a tool first and foremost for the developers pretty much narrows the choice to the Solaris Express Developer Edition (SXDE 5/07).

Of course these days I do most of my development on my laptop (good old ThinkPad T43) but the Solaris OS that I have there is kind of clunky. It is a heavily tweaked Solaris 10, which serves more as a proof that it is possible to run Solaris 10 on a ThinkPad after all, rather than a convenient development environment. In short I had all the reasons to upgrade when I settled on a quest for the ultimate development environment based on Solaris OS. The rest is my account of this quest. You've been warned ;-)[Read More]

Today feels good. In fact today feels exactly like the day after the finals when I can finally stop running like a chicken with its head cut off and focus on import stuff (like writing this very blog entry for example ;-)). Because you see, today is the day when two of the projects I consider to be my personal ones have reached important milestones: Sun Studio 12 is now officially released as a full fledged fully supported suite of compilers and tools for Solaris and Linux. And at the same time Project D-Light has made its first public appearance as part of the Sun Studio Express - June 2007 Build. As I said, these two are very personal (which is not to say that I was the only one who worked on them) yet very different projects. The first one (Sun Studio 12 compilers on Linux) started out as a true skunk works and by the time we got any public exposure we actually had a pretty decent implementation already so the only logical question to ask was: when should we ship it? Project D-Light on the other hand got exposed right after the idea for it ripped through the management chain Alien-style at the time where we didn't really have much functionality to write home about. So what do these two projects and Solaris Express Developer Edition (which is supposed to have its next release available any minute now) have in common? The answer is quite simple -- you. Yes -- YOU, our users, early adopters and loyal customers. Without your support nothing of what makes me proud today would have been possible. Not Sun Studio 12, nor Project D-Light. I wish to thank you and once again ask for the only thing that matters: your feedback. Sun Studio 12 and compilers for Linux ended up a success in large because you clearly showed your interest at our Sun Studio for Linux Forum. I truly hope that Project D-Light has a chance of following that road to success. The Sun Studio Tools Forum awaits!

P.S. As for Solaris Express Developer Edition I keep entertaining the idea of hijacking it and turning into my third personal project. Although it feels like I might be biting off way more than I can chew ;-)

Quite recently I've had a conversation with a good friend of mine who shares my interest in DTrace and compilers and it seems that we've come up with a pretty neat use for DTrace probes. The idea is to start using DTrace probes for assert(3). You know how, during the development cycle the assertions are kept in the code and when the final build is supposed to happen most of them get deleted by turning off a magic define? Well, with DTrace probes used instead of assert() the beauty is -- they are all turned off by default (no funky defines required) but the dynamic nature of DTrace lets you turn any subset of them on when and only when you really need them. Like at a customer site (with the production code running) to diagnose a particular problem. Neat, isn't it?

It is simple enough, really – both were ideas that sounded nice in theory but came crushing down during their first serious brush with reality. And since there's no shortage of experts trying to explain how come communism wasn't meant to be – I'm going to focus on the other one. But before I do, I want to make it extra clear that this article is about dynamic linking and not dynamic loading. The later one consists of dlopen(3)/dlsym(3)/dlclose(3) and is a fine idea. Dynamic linking on the other hand is all about magic that makes your application work, even though bits and pieces of it might be in places you've never heard of. And, of course, as any magic it does promise a lot. Among the biggest claims of the dynamic linking (as it is currently implemented in UNIX and similar OSes) are the following three:

1. all applications are capable of sharing common code at runtime, thus reducing total memory footprint of an entire system
2. all applications can reference common code without actually storing it as part of their ELF (or similar) file image, thus reducing total storage footprint of an entire system
3. you can fix problems in common code, thus benefiting all of the applications available on your systems at once

May be there are others, but these three are most commonly cited in favor of justifying the mind boggling complexity of the modern dynamic linkers (and if you don't believe me how complex they are – try asking our resident Solaris linker guru) and even bigger complexity of how, what I referred to as, "common code" is supposed to be packaged and delivered in order for the magic to work. Of course, given the price we pay in complexity I would expect dividends to be quite significant. Unfortunately, they are not.
The rest of this article discusses why dynamic linking does not deliver on any of its promises and why, just as communism, it might be an idea that only works in an ideal world (as a curious footnote I must add that just as communism done right seems to be doing quite well in one particular eastern country, dynamic linking within a binary compatibility guarantee of one particular OS gets as closed to being true to its promises as one can get).
The goal of this article is not to present an alternative model (I still don't have a 100% satisfactory one even for dynamic linking; not to mention communism) but to merely make the reader question whether static linking is, in fact, long dead and buried or may be the people who try very hard to make us think that way have just spent too much time in an ivory tower and haven't seen the real world in a while.
With that, let me start with tackling the last purported benefit of dynamic linking (an ability to fix problems in common code) not only because it is the easiest to knock down, but also, because once knocked it virtually eliminates the first two benefits completely. An ability to fix problems in common code and effectively addressing it once instead of doing it as many times as you have applications sharing that common code sounds really nice till you ask yourself -- what is a "problem"? What is a bug? And could it be that one application's problem is something that a second application depends upon in order to work properly? The answer to the last question is a resounding YES and there's no better example than a very prominent C++ compiler vendor who had to leave a pretty nasty C++ ABI bug unfixed for a number of years just because any possible fix would break all previously compiled applications. And of course, since C++ runtime library is dynamically linked into any application written in C++ that was unacceptable. You see, in real world programs have bugs. Worse yet -- the line between a bug and a feature sometimes gets quite blurry. That is especially true for common code. Why? For two obvious reasons: first of all, since most likely than not you didn't write the code shared by different applications yourself you have no way of knowing whether your usage patters of that common code do indeed trigger a bug, or whether they are just an example of GIGO principle. Second and most importantly -- you are very likely not to have any control over the common code and even if you can prove that the problem is indeed a bug you'd rather workaround it than wait for a vendor to issue a patch. These two issues combined create a very unpleasant situation where problems in common code now become unfixable not because we can't fix them for good, but because the old buggy behavior is now something that quite a few applications depend upon. This is a classical "doomed if you do, doomed if you don't" principle at work. But where does it leave us as far as dynamic linking goes? In a mess! And a big one at that. All of a sudden we have a system where half of the applications want that piece of common code fixed and the other half wants it broken. All of a sudden we have to make sure that we CAN isolate applications that still depend on an old buggy behavior and the magic of dynamic linking just starts getting blacker and blacker with abominations like LD_LIBRARY_PATH and DSO symbol versioning. What we've got on our hands now is a simple situation where common code becomes segmented in a sense that it is common among just a subset of applications. And that is the point where dynamic linking just breaks. There's no way for my application to be sure that the same common code I tested it with is the one that is being in use. And for any serious software vendor that is just unacceptable. You see, serious software vendors care about their customers and they don't play finger pointing games saying things like: it is all your fault you should have not upgraded that shared library over there. What do they do instead? Well, just try to do

find . -name \.so

to see for yourself. If you do that with any commercial piece of software (or even free large ones like Open Office) don't be surprised to see things like private versions of glibc.so being repackaged and redelivered. It is much safer for them to do that instead of constantly dreading the ugly upgrade of /lib/libc.so.
But wait! Haven't it just annulled the first and the second claims that the dynamic linking had? Sure it did. There's no sharing possible between /lib/libc.so and /opt/bigapp/lib/glibc.so. None. The memory gets wasted as much as the diskspace does. It might as well be static linking at that point.
In fact, static linking would be quite beneficial for the application since if done right with the smart compiler it would enable things like: not wasting precious CPU cycles on position independent code (if you think PIC is free see Performance Analyzer in action), doing interprocedural optimization, cross-file inlining and template elimination. And a few others. And unlike dynamic linking you can be dead certain that the very same code you tested would be working at your customer's site. Not only that -- but when you do need to fix it, you fix wouldn't break anybody else.
Ain't this the magic?

Kuldip has the latest scoop! And for even more details make sure to login
SDN Channel

If you're the type of a person who played a fair share of arcade games in the past AND at least once the code you wrote drove your coworkers to the brink of insanity you should get a kick out of the thingy a bunch of us here at Sun put together just to show how much fun Sun Studio and Solaris can be. So there's fun, fame and $5000 fortune for anybody who dares to enter
Temple of the Sun!