Amiram Hayardeny's My China Experience

Visit To My Home Town - Tel Aviv

My Chinese colleagues taught me many things.  One of them is the love, respect and deep feeling they have for the town in which they were born, the town from where they have their childhood memories.  Their Home Town.  Years, sometimes decades after they have left, they still remember their home town fondly.  I do too, although, I must admit, I never thought of it this way.  Maybe because I have a few of them... 

Yet, talking about the town where I was born, where I have my first memories from, this town would have to be Tel Aviv. 

I'm in Tel Aviv, my home town, and I feel like a stranger.  I will try to put it in words, but I'm not so sure that I'll be successful.  In Beijing, when I walk the streets, or drive around, I get a feeling of people.  Many of them.  In a very special, connected way.  People are riding bikes, driving their cars with the windows rolled down, and simply walking.  In Tel Aviv, like other places, I see no bicycles, and the 4x4 cars' windows are rolled all the way up.  The few who actually walk around in the heat of 35 Celsius, seem to be the ones who must do it.  Tel Aviv is a very young city, or so it seems.  The average face in the street is in its twenties.  Unshaven, tank topped.  They seem that they are on their way to a party or coming back from one.  It is completely different than when I was growing up.  I don't like it anymore.  And then it hit me.  It's exactly the same.  What's different is me.  I'm not so young anymore, if I wore a tank top, police could get involved, and I shave every day.  I am not partying anymore, I am here to see friends and family.

The most significant part was the driving.  I always was a good driver.  After over a year of non-driving life in Beijing, I am more hesitant, I brake more often, I find myself stopping at green lights, just in case someone will come from the other side of the intersection.  I became a Sunday driver...

I met quite a few people over the course of my life and career, who said that going to Israel is a dream they have.  That they would like to visit at least once during their life.  Even mildly religious people, even ones who aren't religious at all, recognize the magic and the spirituality of Israel.  Some pretty interesting, not to mention big time celebrities have lived here.  Flesh and blood, and very well documented.  Indeed, some of the early ones, like Moses and Joshua are borderline myths.  Yet, it is unquestionable that Jesus walked this land, so did King David whose Psalms are still read on many occasions by Jews and Christians alike.  Like King Solomon, the wisest person who ever lived.  People would love to come here, to walk the paths walked by Jesus and dip in the river in which John the Baptist did.  But many of them don't.  The reason is simple.  In fact there are a few reasons, and they all three letter words: CNN, CBS, NBC, ABC, FOX followed by the word News.

I also get my news from CNN.  And when I land in Israel, it is quite a surprise for me to realize that my airplane actually landed safely, and that no particular extraordinary activity was taking place.  I then drive to my parents, and again, am thankful for making it there alive.  And then I realize, you cannot make your travel plans based on a three letter TV news station.  It is safe here for the most part.  People don't hide in their homes, and there is no military activity in the streets.  Everybody is going about their business as usual.  Because this is what beating terrorism is all about, is it not?  Not changing one's life style.  Because if you do, then you lose.  If you stop doing what you want to do, and instead hide, and become scared - then terrorism wins.

So while there is no guarantee, my observation is that whatever it is, it looks a hundred times worse on CNN.  Not only here, but everywhere.  The reason for that is simple as well: bad news sell better than good news.  I wish some American Billionaire starts GNN soon.  Good News Network.  I would sign up in a minute.

Posted at 03:47AM Jun 30, 2007 by Amiram Hayardeny in Personal  |  Comments[24]

早上好, 大连

Me:  "早上好, 大连" (Zao Shang Hao Da Lian, Good Morning Da Lian)
Audience: loud applause
Me: "我很荣幸能代表SUN参加这个盛会。在我的演讲里，我希望你们能找到一些有趣的话 题。并且，我会满足你们的好奇心。如果你们认为我会用中文做这个讲演，那你们错了" (Ladies and gentlemen, how are you?  I am honored to represent Sun in this great conference.  In my speech, I hope you shall find some interesting topics.  I hope to satisfy your curiosity.  If you think that I will deliver the rest of  my speech in Chinese, you are wrong...)
Audience: laughing out loud
Me: “I am a Solaris salesman”.
Audience: murmur
Me: “I came here to sell Solaris to you”.
Audience: loud murmur, one person in the back gets up.
Me: “OK, I lied. I am not a salesman, I am an Engineering Director for Solaris development in Beijing, and I can't sell you Solaris because it's free”.
Audience: relieved, laughing out loud.

This was the beginning of my speech in China's first ever OpenSource conference, which took place on June 22 in the beautiful shore city of Dalian, east of Beijing.  (Dalian).

Once I got the attention, I started talking about Solaris.  It is quite an interesting challenge to be a Sun executive and talk about Solaris, and OpenSolaris.  When I say that Solaris is the greatest operating system on the planet, I become the immediate suspect.  The reasoning is simple: "of course you would say that, this is your livelihood, you get paid to say it".  When I say that Solaris is free, I become even more suspicious: "OK, suppose it is free as you say, what's in it for you?".  You have to admit, these are not simple arguments.  They are not so easy to challenge. 

Yet, I believe I can.  I have been working with all existing OSs since my school days.  That includes Control Data, IBM CMS, MVS, AIX, OS/2, Mac OS (System 7), VAX VMS, MS DOS, MS Windows 3.x and forward all the way, and yes, of course, Solaris.  I truly believe that Solaris is far better than all the above.  I will not get into details, but as a long time IBM Microcode developer, I saw nothing but great respect for Solaris, always.

And what's in it for me?  simple.  The more developers use Solaris, the more applications are deployed on Solaris, the more available the network is for everyone, the more participation in the internet, the more devices, the more activity - the more business there is for Sun.  This model works!

I spoke about Sun's mission: The network is the computer, we want to see everyone and everything participating on the network.   We will create the technologies, and fuel the communities that enable sharing and participation.  We want to eliminate the digital divide.  And we, Sun, put our money where our mouth is.  Sun has contributed $500M or 51,372 person months to Open Source.  Far far more than the second place.  Sun is number one in contributing to OpenSource.  This is widely accepted as fact by everyone I spoke with in the last few days.

I explained a little about CDDL, and its advantages for developers and communities.  I said that in short, CDDL pretty much lets people have the source code, allows identified modifications, demands continued sharing, but asks for no royalties, while giving IP protection.  Who can ask for more?  I mentioned many times: "when I finish my speech, you can log in to the internet and download the entire code base of Solaris onto your laptop.  Free of charge.  You can make changes, build and deploy, free of charge.  You can put it in your production environment, free of charge".  It is yours.

I spoke about record breaking performance, DTrace, Virtualization (Containers, Zones, Branded Zones), Predictive Self-Healing, Security, ZFS, more.  When I finished, I believe that there were a few hundred people who became more aware of Solaris, its price, it features, its communities.  They became more aware of Sun's commitment to OpenSource.

The afternoon session was different.  A few hundred students, apparently freshmen, were in the room, and I was asked (by complete surprise) to deliver an introduction to OpenSource (at least that's what I gathered before going to the stage).  I told them about my first programming assignment, in my freshman year (thanks for the interpretation, Kevin!).  Take as input three parameters, and return whether the three numbers constitute a triangle.  I asked if anyone knew the algorithm, there were many hands up in the air, one student won a T-Shirt...  I told them that I wrote my assignment in Pascal, on a Televideo terminal (green on black), connected to a Control Data Cyber 170, and completely alone: no internet, no community, no help.  I told them, that they live in a completely different era.  They are so connected.  I told them that if they run into a difficult programming problem, or a bug, all they need to do is shoot a message to the community, and in no time they will get answers from tons of people they never met, in places they never heard of. 

I told them that they will change the world, and that Sun will give you the tools to do it.

Audience: loud applause.

Posted at 11:01PM Jun 22, 2007 by Amiram Hayardeny in Personal  |  Comments[2]

Memoirs of a Taste Bud

In my mind, Israel is the absolute best place in the world for dairy products.  A large variety of Cheeses, yogurts, cream cheese, yellow cheese, smoked cheese.  Cheese lovers' heaven.  Indeed, there's nothing like Italian Parmesan, Romanian Kashkaval, or Greek Feta.  And undoubtedly, the French make the most stinking cheese there is.  In fact, when you open a package of Morbier cheese, it smells like something that died behind the refrigerator and the maid couldn't get it out...  Israeli dairy products are special because they blend the flavors of European dairy products with the Mediterranean and Middle-Eastern ones.  Goat and sheep cheese (even camel if you insist), spices, and textures.  In short, get a Russian dark bread, some butter, and a large slice of Kashkaval cheese, and you are all set...  You can get almost everything here in Beijing.  With few exceptions.  But the only kind of cheese that we really miss here, that we weren't able to fine is the creamy low fat white cheese.  One time we visited Carrefour, and we thought we found something similar, but it turned out to be, well, not even close.

Frankly, I haven't seen this kind of cheese anywhere on the planet.  (Don't confuse it with the sour goat cheese - Labane...).

Fresh bread, great butter and cheese helped me become what I am today.  In American there are some nice words to describe it: big, heavy, sizable, chunky, full-figured, round around the mid-section and so forth.  But lets face it, I am fat.  The bottom line is, I must lose weight.  My wife, with her infinite wisdom and patience, ships me to work every morning with the same container of vegetable salad, with some tuna and some yogurt on top.  At 12:00 noon, precisely, I go to the refrigerator, open my container and look at the daily surprise.  I don't really get disappointed when I find out it is the exact same lunch as yesterday.

Anyway, if you think that eating boat loads of vegetables made me lose a pound, guess again.  In fact, I am afraid that I have found out the well kept secret of the elephants.  They eat lots of vegetables, and they remain big, really big.  I feel the same.

But never mind that.  The point of this entry, is that after a few days of having the "daily surprise" for lunch, I recognized a pattern.  The yogurt on the top of the salad changed its original texture, and the bottom of the container had a half a cup of water.  And then the light bulb lit on top of my head.  If I could find a way to get rid of the excess water in the yogurt, then the leftover will be exactly what I was looking for: my creamy low fat white cheese.

We bought a few containers of yogurt.  We took a pot, and placed a couple of pages of paper towel inside a pasta drain, then poured the yogurt into it and placed it in the pot.  24 hours in the fridge, added some salt, and we had a delicious creamy white cheese.  You can add olives, scallions, smoked salmon, even walnuts or anything else your heart desires.  Enjoy!

 
 

Posted at 01:56PM Jun 19, 2007 by Amiram Hayardeny in Personal  |  Comments[1]

Sunday Morning at the Beijing Zoo

We took the kids to visit the Beijing zoo this morning.  As an animal lover, going to the zoo always presents a dilemma.  On one hand, it's hard watching the poor animals locked in their little cages, far from their natural habitat, completely dependent on their human keepers.  On the other hand, where else can you show your children real live animals?  So we go, and feel sorry, and overcome it, and move on.  Yes, there are other possibilities.  The Serengeti National Park in Tanzania is incredible (so says my brother-in-law, I've never been), but it's way too far and way too expensive.  The zoo in San Diego is spectacular.  But in general, wherever animals are locked up in small spaces is not really a happy place.

We still had a good time.  If you are in Beijing, I recommend visiting the zoo.  It's an unusual opportunity to see Panda Bears, and many other animals, birds in particular, that I have seen nowhere else.  I probably wouldn't have written about it, except there was a riddle at the zoo, and I couldn't solve it.  Care to try?  Here's the deal: look carefully at the picture below.  If you can figure out where North really is, please let me know.  All the right answers will take part in a lucky draw.  Winner gets two tickets to the Beijing zoo (Panda display included).  To make it a little less challenging, I marked the most significant clues in red circles.

Posted at 05:14PM Jun 17, 2007 by Amiram Hayardeny in Personal  |  Comments[4]

Thank You Miss Ruth!

Thank you for taking a bright girl, who used to be top of her class both socially and academically, and making her, well, the same.  Thank you for taking this happy, confident, trusting, really amazing child, and turning her into, well, the same.  Maybe not exactly the same, after all, she did travel 7000 miles to get here, and she had to be taught a native language, and she lost her smile in the process...

You, of course, know what I am talking about, but let me explain for the benefit of the others.  We came here on July 2, 2006.  Shiri did not speak a word of English, nor did she speak Chinese.  Overnight, Shiri turned from most popular girl in her class into a mute.  She didn't understand what was going on in class, she couldn't play outside with the other children, she was sad, she stopped laughing.  I'm sure you understand the pain a parent feels when a decision he or she made turns his child's life upside down this way.

But with endless patience, support, encouragement, you took her by the hand and you didn't let go.  And soon enough she started to take her first steps in the new country.  She started to talk, to understand, to participate, her smile emerged.  I am reading her report card with tears in my eyes.  I am amazed at this little girl's determination and courage, at her good fortune for having you, and at the fact that my life would have been different if I had my own Miss Ruth in second grade...

Dorit and I and Shiri of course, owe you big time.  You gave her the gift of knowledge, of curiosity, of academic interest.  You gave her the understanding that hard work pays, that rough periods end, that there's a lot of good in this world.  We shall never forget you Miss Ruth!

Posted at 12:48AM Jun 16, 2007 by Amiram Hayardeny in Personal  |  Comments[0]

Global Warming a Confused Opinion

For years I have been hearing about Global Warming.  Like a band of doomsday prophets, a whole bunch of scientists, reporters, and regular fanatics would show up somewhere, preferably at a G8 meeting, and preach about the environment, recite the "green mantra".  I slowly have reached a conclusion that one of the worst things that happened to the environment, is that it chose not-so-great representatives.  The truth is that I am thoroughly confused now.  I have read Michael Crichton's State of Fear, and I have watched Albert Arnold Gore Jr.'s "Inconvenient Truth - A Global Warning", and I can't tell who is right?, who is wrong?  Will we be here in 50 years?

Al Gore's prophecy is very convincing.  You absolutely have to love him.  He is a very impressive man, with an amazing biography.  Born in 1948, he is a Harvard graduate, a Vietnam Vet, A retired Congressman, Senator, and Vice President.  And he will only turn 60 next year...  He is probably the best presenter I have seen, ever.  He uses his appearance, his voice, wide gestures.  He is armed with the best data available on this troublesome planet.  In the film, he's also using the absolute best presentation equipment: an IMAX screen (or close to it), an amazing resolution projector, sound system, even a crane that actually lifts him up to meet the carbon dioxide line on top of the screen...  But in all honesty, he could have been presenting at a grade school, using a blackboard and chalk, and I would still think he's a great speaker.

Michael Crichton is not very far behind.  Well maybe far enough, but still a front runner compared to the rest of us.  Also a Harvard graduate with an MD degree, and a long string of extremely successful best sellers behind him (I read them all, started with the Andromeda Strain long before it became a best seller)..  Crichton is also very convincing when it comes to provide enough data to suggest that the "green" guys, the "environmentalists" are, how to put it, not telling us the entire story.  They use the well manipulated data to draw conclusions that simply cannot be drawn from the data we have.

As a result of this confusion, I decided to take a completely different look at it.  My very basic, almost ignorant assumption, is that "something is going on".  Apparently the population growth, along with the very fast technological progress we have been making and the increased burning of coal and fossil fuels have been making some kind of impact on our habitat.  I decided to stop asking what is the precise affect,  or when, precisely, will Florida be under water, or when, to the day, will Antarctica completely melt.  Instead, I decided to present a different question.  Can we afford to take a chance?  What's at stake here?  If you look at the following link, you shall realize the nature of the argument.  Was 2005 really the warmest year ever, or does 1998 still hold the record.  (http://www.livescience.com/environment/060201_temperature_differences.html).  It's like an argument taking place at the ER, when a severely injured person is hauled in.  "Did he lose one litter or 1.7 litters of blood"?  The answer is simple: if he doesn't get a blood transfusion within the next minute or so, it won't really matter, now would it?

Suppose we decide to ignore the warnings and just continue with our business of deforestation and burning of fossil fuels and find out fifty years from now that the "fanatics" were correct, and that the changes are far too significant to reverse?  The opposite should also be asked: what if we take all the right measures: use wind and solar power, burn natural gas instead of fossil fuels, recycle, and wake up in fifty years only to realize that the threat was empty?  The ice cap was not in danger, the planet was doing fine...  Can you imagine waking up to a cleaner world one day, realizing that we did all that in vain?

And what if we actually find out that the damage will become irreversible in 100 years, 500 years, 1000 years.  Should we just continue and take comfort that we have a place to live for now, and whatever happens in the future, well, that s the problem if the people of the future...  Do we owe it to our descendants to leave them a habitat similar to the one we have received from our ancestors?  If we were to look them in the eye, will we have to apologize?

Indeed, it is amazing that the number one polluter in the world in the USA.  It's amazing that the only two advanced countries in the world not to ratify the Kyoto Protocol  is the US and Australia.  It is outstanding that it is acceptable to us, that for the purpose of supposedly saving a failing car industry, we are ready to sacrifice our homes.

Still, I believe that there's something wrong with the message.  Not with the messengers - they are outstanding, but the message itself is problematic.  Is it possible that there are too many scientists, with too many grants, too much media, too many lobbyists?  Agreeably, where seven scientists convene, eighteen opinions emerge.  The need and urge to be "unique", "special", "creative", the ability to massage the numbers to show a slightly different conclusion, is sometimes interesting.  But when your house is on fire, rather than listen to a hundred scientists arguing about the actual temperature of the core of the fire - I go call the Fire Department...

How do we make the message such that everyone, everywhere, starts to think: we must do something about it.  Today.  Do I know the answer, absolutely not.  Maybe scientists should forget their grants, their scholarships, the papers, and the research, and they should come up with a unified message to say something like: "We are not sure about the precise nature of the phenomenon.  But we are messing with this planet.  And if we don't dom something about it NOW, we will regret it forever, and our descendants will have no place to live".  Anybody?

Posted at 10:39PM Jun 14, 2007 by Amiram Hayardeny in Personal  |  Comments[3]

Blog Appearance Changes and Shining Suns

I am sure that at least some have noticed that I made some changes to the pictures, and to the layout of the different gadgets and buttons of the blog.  I wonder if I could solicit some feedback?  Anyway, on the side column, I placed 5 shining suns.  One large one, and two smaller ones on each side.  I have no idea if anyone tried it at all, and whether anyone thinks it's an imposition.  However, the 4 small suns are linked to YouTube, where you'll be able to see four hilarious commercials from Sun.  I have no idea whether they were ever aired.  But I watched them all, and regardless of whether you like or dislike Sun equipment, software, or services, you will have to admit that these commercials are extremely funny.  The large Sun is a link to a video that I see every time I go to a Sun event like Sun Tech Days, or OpenSolaris Day.  It is a beautiful, exhilarating video which conveys the Sun message with outstanding photography, video and sound.  I would classify it as art rather than advertising.

Having said that, if anyone objects to the presence of these buttons, say so, and I will reconsider...

Posted at 08:36AM Jun 13, 2007 by Amiram Hayardeny in Personal  |  Comments[2]

By Request: DTrace

I have been asked multiple times to describe DTrace, to explain the benefits, the novelty, and the reason why this is such a hot topic in the Unix world in general, and the OpenSolaris world in particular.  I thought it would make an interesting blog entry, since most of the questions came from my close professional community.  As usual, I chose Wikipedia for definition and starting point.  I must add, that finding the definition on Wikipedia pleasantly surprised me, but then I realized that I shouldn't have been surprised.  DTrace is a great tool as well as a disruptive technology, and it deserves its place on Wikipedia (and all the other 1.5 million locations where DTrace is mentioned on the Web...).  "DTrace is a comprehensive dynamic tracing framework created by Sun Microsystems.  It was released under the Common Development and Distribution License (CDDL) in January 2005 and included in Sun's Solaris 10 for troubleshooting system problems in real time. DTrace was the first component of the OpenSolaris project to be released under the CDDL."

Before I try to explain what DTrace is, the way I see it, let me present some history and explain some terms.  Naturally, software developers always used novel ideas to make better software, to debug more effectively and efficiently.  A debugger would be the obvious choice for an application level programmer.  But what about the Kernel developer?  What about systems executing in really hostile environments like multi processor, shared memory, multi threaded, high volume transactional systems with little or no room for failure (five nines) - system must be available 99.99999% of the time?   In these environments, developers commonly use logging or tracing.

• Instrumentation
• "The process of inserting trace statements into source code, which when compiled will write trace data".  As you can tell, this is a very static statement, and it needs a lot of (a) planning - to understand the nature of the failure, and to know which are the correct source code elements where to insert the trace statements., and (b) trial and error - sometimes dozens of trials (and many more errors) are required to find the right trigger for the failure combined with the right trace statements.  In high end servers, it may take a full day per cycle to build the package, load it, boot, set up the suspicious test case, and then waiting for it to fail.
• Software Tracing
• Software tracing is very similar to logging.  Every function entry and exit is registered and kept in a log (usually a file on disk) with some hopefully relevant debugging information.  It is possible, with tracing, to understand the flow of execution, the order of function calls, the executing CPU and values of local and global variables.  Tracing, or logging is relatively simple: in every function, an OS service is called to register the entry.  It does the same when the function loses the CPU (by being preempted or by finishing naturally).  Within the function, the developer defines situations which are "debugginly interesting:, and calls an OS trace function to record them along with some debug data. 
  
• There are multiple drawbacks using this system
• Trace buffer size
• The quantity and the quality of the data are almost mutually exclusive.  Usually, the trace buffer is of fixed sized.  Therefore, the more trace information is recorded, the less time of execution it represents.  Once the trace buffer is full, it wraps around.  I have seen high end servers, with a huge trace buffer, which could only record a fraction of a second of execution traces.  If a decision is made to record only the most critical debugging data, the trace buffer may reflect a longer period of time - but with less relevant data.  In other words: chances are the data you absolutely need for debugging will be missing...
  
• Analysis is done on a static dump which was taken minutes, hours and in extreme cases - days after the failure, with little or no relevant debug data.
• Changes in tracing must be done in the lab, on a special build.  It can't be done on a customer machine, with the customer's build, and the customer's pattern of execution and workload - which in many occasions has a lot to do with the failure.
• You can't do debugging on a production machine running production software and production data.
• In order to get better dump data, you have to have more trace points, a larger trace buffer.  Otherwise, you can't "catch" the failure.  But all the excess trace buffer entries, and excess logging activity, may change the timing of execution and sometimes prevent the failure from occurring at all!  It may also affect the performance significantly.

In general, the process of troubleshooting system problems in real time using tracing or logging is along the following lines:

• Repeat until success
  
• Pack the suspicious source code module with trace entries (for logging purposes), make sure that they contain information about the function, the value of some variables at certain points during the execution.
• Generate a new package, load it.  Boot the machine and set up the test.  Then, with some hope, and luck ("We're going in on a wing and a prayer", as one of my old colleagues used to say), the failure would happen at the right moment, and the debug data will be readily available.
  
• Increase the size of the trace buffer (as much as possible)
  
• Execute the program.  Trigger the failure, get the dump on a disk.
  
• Analyze the trace buffer using some tools, which basically synchronizes the execution in time continuum (absolutely necessary in the case of multiple CPUs)
• If you have the root cause - kudos!  Well done! Exit
  
• Make adjustments to the trace calls in order to get closer to the root cause.  Back to "1"
  

Troubleshooting system problems is not a very special thing.  It's been done for years.  What makes DTrace unique is the fact that debugging with DTrace can be done in real-time, on production software and data during regular execution, with much less special treatment to the environment of execution, and with many more cycles per time unit.  Add to that the fact that DTrace can be used to fine-tune systems for performance, and you got yourself a winner.  This alone, in my mind, answers the question of why the topic of DTrace is so hot.

What about DTrace (Dynamic Tracing)?  What makes it so special?

• The DTrace framework provides instrumentation points that are called probes.  A DTrace user can use a probe to record and display relevant information about a kernel or user process. 
  
• Each DTrace probe is activated by a specific behavior. This probe activation is referred to as firing.  Consider a probe that fires on entry into an arbitrary kernel function. This probe can display arguments passed into the function, values of global variables, timestamps, stack trace which indicates the flow of control up to that point of entry, and a lot more.
  
• Note that with DTrace, there will be significantly less planning and "trial and error" cycles.  Also, the "trial and error" cycles which must take place will be an order of magnitude faster, since the need to build, package, boot, and set-up tests is eliminated or significantly shortened.  Even on high end, multi-processor, multi-threaded, high volume transactional systems - it strikes me that rather than the traditional one or few cycle per day, one can run dozens per day, possibly root causing defects in a fraction of the time.
• Note further that bug fix verification time can also be reduced significantly using DTrace.

How does it work?  Remember the example probe from before?  The one that fires on entry into a kernel function?  Imagine that DTrace can dynamically enable and manage thousands of probes.  DTrace will allow you to write new probes, associate them with others, and conditionally fire them.  Trace buffer management is also done dynamically.  DTrace provides the ability to examine trace data from a live system.  The framework allows not only for new probes to be implemented but also for new ways to present the debug data.

To sum it up, DTrace will allow you to set up a fully instrumented debugging environment.  You will save much time (and resources) in setting up, making, packaging, booting and test environment set-up.  You will be able to do it in the customer's data center if necessary (and if the customer will allow it), with no performance degradation or increased chance of failure.  You will be able to do it all dynamically: launch probes and view their output while the program is running, create conditionals where some probes will fire, and others won't.  Think of it as a real-time positive feedback loop of debugging: fire a probe, see the results, fire a closer probe, see the results, identify the point of failure, devise a fix, test and integrate.  All dynamic, all in record speed.

Ah, and one more level of uniqueness: it is not available for IBM AIX, Linux, MS Windows.  Introduced in Solaris 10 and released to OpenSolaris under CDDL, DTrace is a Sun thing, and it is available free of charge when you use OpenSolaris.  As a former microcode developer, I can only think of the sleepless nights I could have avoided if this tool was available to me ten years ago...And I wouldn't even mention a certain gold medal given to Sun Microsystems Inc by the Wall Street Journal in its 2006 by the Technology Innovation Awards.  It was given for DTrace trouble-shooting software.  Well. I did.  Wouldn't you?

For more information:
http://docs.sun.com/app/docs/doc/817-6223
http://uadmin.blogspot.com/2006/05/what-is-dtrace.html

Posted at 07:28PM Jun 12, 2007 by Amiram Hayardeny in Personal  |  Comments[0]

Spam for Thought

My wife gets scores of email on a daily basis.  Most of those are not work related, they don't include stock market analysis or urgent medical data.  In fact, judging from the samples Dorit is kind enough to forward to me every so often, they are the best examples of spam mail.  All of us, members of the Internet club, must go through spam emails regularly, and devise ways to be less exposed to it.  Every time we come up with a way to see less spam mails, someone comes up with a better scheme to force it on us.  So what is spam mail?  What drives it?  Who makes money of it?  Are there ways to stop it?  What are the legal aspects of spam mail?  I know it's a big issue, and I must add a disclaimer: I am not going to try to cover the entire topic, it is way too big.  I will try and touch on the points that bug me, and most likely - you.  Also, spam, or unsolicited messages, comes in much more forms: regular mail, fax, voice mail, and other shapes as well.  I will only refer to the form of spam that comes into my electronic mailbox, every day, all day, seven days a week, year round.

Lets start with the term "spam"  where did it come from?  What does it mean?  SPAM is known to me as canned meat from Hormel Foods.  How did it become the accepted term for unsolicited emails?  Apparently, since SPAM appeared in 1937, it has had a long history of repetitive marketing which practically put SPAM in almost any food you can imagine: SPAM sandwich, SPAM and eggs and even SPAM muffins.  There was SPAM omelet, fried SPAM, stuffed SPAM.  SPAM was everywhere.  According to Merriam-Webster Dictionary, the etymology of the word "spam" in relation to unsolicited, usually commercial e-mail, sent to a large number of addresses, comes from a skit on the British television series Monty Python's Flying Circus in which chanting of the word Spam overrides the other dialog. (Spam: definition).

The following is the statistics of spam mail, and its growth over the last few years particularly.

• 1978 - An e-mail spam is sent to 600 addresses.
• 1994 - First large-scale spam sent to 6000 newsgroups, reaching millions of people.
• 2005 - (June) 30 billion per day
• 2006 - (June) 55 billion per day
• 2006 - (December) 85 billion per day
• 2007 - (February) 90 billion per day

It is estimated that 80-85% of all email is abusive - unsolicited and unwanted.  Amazingly, as I was reading about spam, I ran across this: Most spam contains a URL to a website.  According to a Commtouch report in June 2004, "only five countries are hosting 99.68% of the global spammer websites", of which the foremost is China, hosting 73.58% of all web sites referenced within spam.  (Wikipedia on Spam).

How much does spam cost us?  The European Union estimated in 2001 that the annual cost is €10 billion.  However, in 2004, the cost estimated for US companies only came to $10 billion in 2004, including lost productivity and the additional equipment, software, and manpower needed to combat the problem.  Judging from the table just above, the mountain of spam email grew threefold since 2005.  I am assuming, conservatively, that the cost of spam email grew at least as fast to $30 billion in the US alone.  I am saying conservatively because chances are that the tools, server, electricity, and lost wages also grew at the same time, and the total amount is actually greater.

What is it that we're getting against our will?  First and foremost, products advertising 25%, Financial 20%, Adult content 19%, Scams 9%, Health and Internet at 7% each, Leisure 6%, Spiritual 4% and 3% of other stuff.

Is it legal? In the US spam is legal according to the Federal CAN-SPAM Act of 2003 (Can Spam Act of 2003), provided it follows certain criteria: a truthful subject line; no false information in the technical headers or sender address; "conspicuous" display of the postal address of the sender; and other minor requirements.  If the spam fails to comply with any of these requirements, then it is illegal.  Aggravated or accelerated penalties apply if the spammer harvested the email addresses using methods described earlier.  Just recently, it was reported that the "'Spam King' arrested, charged" - "Seattle resident Robert Soloway has allegedly sent billions of unwanted and illegal e-mails. He faces 35 counts and decades in prison, according to a newspaper report."  (Spam King Arrested).  Will this arrest clean our mailboxes?  Direct less unwanted emails to our electronic mailbox?  Most likely - no.  In fact, it is most likely to increase.  The natural question is why?  How can someone make a living off spam email?

I always thought that the way it works is to employ the law of truly large numbers, which claims that with a large enough sample many odd coincidences are likely to happen (Law of Truly Large Numbers).  According to this law, it is very unlikely for one person to buy a product or service advertised by unsolicited mail (electronic or other).  Yet if you send to a billion people, the coincidence of finding one or more fools to buy your stuff is more likely.  All you need to make sure is that your cost of sending a billion emails is lower than the profit you make off that sale.  The words "your cost" are key here.  It is quite possible if the real cost of sending a billion emails is calculated, it surpasses the profit made from the one sale in a very significant way.  But if someone else pays for it: in network bandwidth, servers and other equipment, waste of wages and time - then why not so it?

As it turns out, my assumption was very limited.  One spam business model is as follows.  The spammer will get a fraction of a penny for every web page viewing.  You, the victim, don't have to register to anything.  All you need to do is click, enter a web page, and the spammer gets his fee.  This has to do with the law of large numbers (not the truly large numbers).  The more people enter the web site, the more pennies collected by the spammer.  A business.  In fact, you may want to view the web page, so you can "unsubscribe" from the "spam service".  Don't.  The spammer will get paid, and you will get thousands of more messages...  There are the fraudulent spammer, who will try to either sell you something you will never receive, or convince you to pay a ridiculously small amount of money in hopes to win big money.  No worries, nobody ever won anything, you are not alone...Others are after your identity, credit card numbers, other things.

The safest thing to do with a piece of email whose origin you don't recognize, is delete it.

Having said that, I admit that sometimes I get these emails, particularly with .ppt attachments,, and I do open them.  And invariably, the presentations always show some deep insight and understanding of life which I wasn't able to gain before.  Some have great photos, real corny stories.  Occasionally, they do actually strike a chord.  As in the law of the truly large numbers...

Lastly, another important disclaimer: spam email has absolutely nothing to do with SPAM Luncheon Meat that we all know and love (?). 

Posted at 11:27AM Jun 09, 2007 by Amiram Hayardeny in Personal  |  Comments[1]

Artificial Evolution and Software

How did chickens lose the ability to fly?  How come some dogs are docile, quiet, "great with kids", while others are aggressive, dangerous, unpredictable.  How were cows domesticated?  Horses?  Pigs?  Why are people so set to get a "pure bred" dog?  Why is regular rice so different than "wild rice"?  The answer to all these questions is "Artificial Evolution".  Evolution, in a nutshell is the following process: a species is content living in some environment.  One or more individuals come up with a spontaneous mutation of some characteristic.  Nobody cares, nor does it bother the individual.  Until the environment changes in a way that makes this individual, along with its spontaneous mutation to become more comfortable (adapted, successful, ...) to the new environment.  Over time, this individual may get a better chance at reproduction, hence increasing the frequency of that mutation in the population.  Over a longer period of time, the entire population may carry that mutation.

Note that this description is slightly different from the description of the "cataclysmic evolution", in which the one with the mutation survives, while all the rest of the population die while suffering great pain due to the new environment.  I can't avoid writing about an argument I have had with a very close person who was absolutely positive that the giraffes got their long necks by stretching it to reach the leaves on the trees at a time of famine.  The ones who did that survived and their offspring inherited this convenient characteristics.  Obviously neither are correct.  Changes must be genetic to survive reproduction, and in the case of cataclysmic evolution - everyone dies and there is no chance to pass on anything to anybody...

The question stands: how does domestication occur (in animals as well as plants)?  The joke says that for humans the answer is rather simple: they get married...  But jokes aside, how does a wild animal become docile?  The answer is evolution, of course, but in this case not natural evolution, but artificial - man made.  Lets assume for the sake of the argument that you have a couple of sheep that you caught while hunting.  They are aggressive, violent, and you run the risk of losing them all the time.  They reproduce, and the litter, while very similar to the proud parents, have one, particularly quiet, non-aggressive individual.  You realize that, and when the litter matures, you prevent the others from reproducing, while making sure that she does.  The new litter has a far better chance of being less aggressive, easier to handle.  When you do it over a few generations, you will end up with a calm, docile, domesticated herd of sheep.  Same goes for cows, dogs, pigs, etc.  It isn't that different in plants.  You look for the corn plant that gives the most cobs, the better grains, and you make sure that it is the only one whose seeds you use for the next season.  Keep doing that, and you have a better corn.  That's the way it's been done for centuries.  That's how we have high-yield milk producing cows, ridable horses, calm sheep, hunting dogs, etc.

It also goes the other way.  If you select the absolute most murderous Bull Terrier in the litter, and let them reproduce for a few generations, you will get the American Staffordshire Pit Bull Terrier, who is anything but docile, quiet or "great with kids".  If you pick the tomato whose shelf life is the longest, and use its seeds only, you will get a species of tomatoes which can sit in the supermarket for years, but taste like rotten styrofoam...

You could, I guess find similarities in the world of computers.  An application is released.  The "litter" could be millions of copies.  Developers select the good, most successful, most adaptive modules and evolve them to be better by fixing bugs, adding features (hence, in a way, letting them reproduce).  The modules which are not successful are left behind.  Over the course of time, software (hardware too) evolve.  Malicious developers take the worst characteristics of software, and let them evolve to create more aggressive, violent, murderous software modules: viruses.  It's all about evolution.  Artificial evolution that is.  Keep in mind, that the market always welcomes the first kind, the kind that produces better software, better quality, better features.

 

Posted at 10:12AM Jun 06, 2007 by Amiram Hayardeny in Personal  |  Comments[11]