..bits & bytes teleported

I am going to graze in another pasture now. Today is my last working day at Sun Microsystems. It was one big party. I love Sun evermore now.

Thanks for all the fish. If you wish to get in touch, please email: amitsaha.in@gmail.com . I currently blog at http://amitksaha.wordpress.com

Adieus.

This blog entry is directed to someone with skills and interest in NetBeans APIs- module and platform development and Python.

I have been working (in my spare time) on the Python support for some time now. Recently, the EA2 release was out and though the nbpython@netbeans.org  alias is not, what you would call high-volume, but there is pretty good interest and satisfaction among the users. This  has spurred and inspired me to keep working on the project.

I would like to bring to your attention and hopefully guide you to some very low hanging fruits (IMO) that you could grab and help us with:

• Online Help: Online Help constitutes the help sets that is shipped with the IDE. I wrote the current stuff long time back. They really need some updation, since a lot of enhancement has been done since. You will need some knowledge of JavaHelp to start working with them. But, its easy. You should take a look at the python.help module in the NetBeans sources. 
• Documentation: Besides the IDE Helpsets, we also need some 'web-based' documents which would target documenting a specific Python development scenario in NetBeans. Some examples: this, this and this. Some of the docs that we would like to see are summarized here.
  
• Issues or Bugs: There are a *lot* of Python specific bugs.Not all of them require much or very less coding exercise. Some would possibly be just verified and closed. You could take a look there. Some are Enhancement requests which would be a good exercise if implemented

That's some of the very easy ways in which you could help the Python support project. Of course, you could also scratch your own itch.

Talk to us at nbpython-dev@netbeans.org  and nbpython@netbeans.org

The winners of the second Code For Freedom contest (organized by Sun Microsystems, India) have been announced: http://in.sun.com/communities/univ/codeforfreedom/winners.jsp A very hearty congratulations to all the winning students and colleges. 

A special thanks is due to Sonya Barry, who helped us relentlessly with getting all the projects on http://dev.java.net. You may view all the projects carried out as part of the contest at http://codeforfreedom.dev.java.net

Hearty congratulations and much thanks is due to all the engineers in Sun India who were involved with the contest in some form or the other- project evaluation, poster presentation, answering students' queries and so on.

On a very personal note, it is a great experience to be able to manage and help run such contests, especially when I was a participant in the first edition.

We still have the presentation ceremony coming up on18th July in Bangalore.

Congratulations one and all. May the source be with you!

The DTrace community doesn't really need another blog post to introduce the technology to folks who are into this kind of a stuff. Why this blog post then ? Please read on. (Note that, this is strictly for newbies of various things)

Scenario #1: I had to explain DTrace to my friend who was going to talk about some JVM + DTrace stuff in a presentation in her college. I suggested her the topic. Now the hard part was to even explain the basic idea behind DTrace and convince her enough so that she in turn could convince her evaluators- both parties being almost completely ignorant about DTrace. 

Scenario #2: I have been exploring a bit about MySQL + DTrace and was going to talk on it at the Bangalore MySQL UG meet yesterday. As expected, the audience was completely ignorant about DTrace and perhaps, of more importance- all of they ran MySQL on Linux for their business. Ah now things gets interesting. I wanted to 'wow' them with DTrace

One common thing between these, is that the audience in both cases are DTrace newbie. Before delving right into what DTrace can give us with all the technical details of probes, providers, consumers, production systems, et cetra , its very useful to set the stage with some real life analogies is set for the audience to move on to the excellent technical manuals available out there.

The remaining part of this post has the text from which my words were derived in both cases:

To introduce DTrace, you will have to speak about the importance of run time observability of production systems. Perhaps, that sentence was not very informative, however, please stay with me. I shall explain it.

Production systems:  We call a production system as such, when it is used to provide any kind of consumable service to consumers. Consider a scenario: the billing system in a departmental store serves all the billing terminals- the ones at which you get your stuffs billed. The billing system is perhaps run off a single powerful computer or a network of many powerful computers housed somewhere in the store or more likely in a central location, not in the store. The billing system is an example of a production system. Its in use and you cannot
take it down. If you do or it decides to do so on its own, you will face the wrath of customers and bear losses in your business.

Observability: In simple words, observing a system means to learn to see the various ways your program- a 10 line program or a billing system- is behaving at run time. Now, what do we mean by 'behaving' ?
Let's say, our program is now using 10% of the total CPU and other point of time it is using 90% of the CPU. That's one behavior of the program. More simply, let's say, for input X, the program is in function, F and for input Y the program goes to function, G.

Observing a system during its run time is hence also called dynamic observation of the system, 'cause it happens in real time. Since we basically 'trace' the behavior of the system at various instants of time, we can also refer to it as 'Dynamic Tracing'.

Off Radioactive elements and DTrace

Tracing of a system is a lot similar to the way Radioactive elements are used as tracers in human body to identify various abnormal body processes. Our topic of interest here, DTrace works a lot like this.

Using the DTrace framework, we can implant probes (or tracing points, like 'radioactive tracers') in our programs and can subsequently monitor those points for activities of interest. What we are interested in here, is Live Analysis and not post mortem analysis

I would also like to suggest you to go through the slides that I used for the talk. The later slides are MySQL specific, but the first few give some general information. It also incorporates some information that I was suggested by Richard on the dtrace-discuss list. Thanks to Michael and James for their suggestions. James also suggested some basic DTrace scripts that I could use to wow the audience. However, I will save them for the getting started session. I have just aquainted them.

Now its time to read http://www.solarisinternals.com/wiki/index.php/DTrace_Topics_Intro

I hope you find this post useful, and appreciate your comments.

The Bangalore MySQL User Group is having a meetup this Saturday- 21 March. Please RSVP if you are coming.

We have the following sessions planned:

1. MySQL Query Analyzer -- How to optimize queries using query analyzer-  by Thava Alagu, Sun Microsystems
   
2. Using MySQL + DTrace -- By Amit Saha, Sun Microsystems, this session will be derived mainly from my article.
   

See you there!

The following blog post and the opinions expresses there in and their effects is strictly limited to the IEC Open Source Mela. No ripples beyond the effect is intended.

The IEC Open Source Mela is just a week away. Apart from being one of the organising team members, I am also in charge of the business plan event- Swatantra Saudagar (two Hindi words, 'Swatantra' meaning 'Freedom' and 'Saudagar' meaning 'Businessman', or a 'Trader').

What is it all about?

Simple: Formulate a business plan using only Free and Open Source software. Your business should stand on Free Software.

Now, you get the idea behind the name ? Oh well, yeah agreed that none of us are Free (Read about मुक्ति or Moksha) but atleast we can use Free softwares to build world changing business models. Making a Business plan is one thing and making a profitable business plan is another. That is the challenge.

Why does Free Software make sense?

The answer is in the question itself. Essentially, almost as a rule, large sums of money can be saved by using legally Free (as in cost) software to start off a business- a product, a dot com or a teaching academy. At the beginning, all such ventures have more time than money. This time can be utilized to tinker around with the tools to fit the specific purpose.

More..

Please read The Business of Free Software

.. is up on MySQL Developer Zone at http://dev.mysql.com/tech-resources/articles/getting_started_dtrace_saha.html

Have a comment on the quality, or the clarity or the usefulness? Let me know!

Many thanks to Giuseppe, Lenz, Martin MC Brown and Aaron for helping in various ways.

As part of Sun's university relations, seven Sun engineers- Vadiraj, Varun, Blessen, Lawrence, Nirmal, Gautham and yours truly went to Vellore Institute of Technology, Vellore during the past weekend for a multi-track technology talks on Java ME, GlassFish, MySQL, JavaFX, Open Solaris Virtualization technologies.

There were three parallel tracks:

• J2ME, GlassFish, MySQL- which form a nice combination of Client, Middleware and Database technologies and our (Vadiraj, Varun and Me) idea was to give hints here and there to the students how to bring all the three technolgies to develop applications and how each fits into the holistic idea. My MySQL slides are available here.
• JavaFX
• Open Solaris Virtualization technologies

It was a hot Sunday, thought not enough keep the ever enhusiastic students in their dorm rooms, ofcourse. All the sessions were well attended and I am sure this Sun Tech Day at VIT will lead to more awareness and adoption of the technologies among Students

It was great to meet folks like Rohit (whom I had 'met a day earlier via Twitter), Aditya Kochar, Ram and few others. These students ensured everything was set up for the day.

It was a nic trip, especially amusing being the fact that there is a functional public railway track which runs through the campus of VIT, Vellore.

Mela is a Sanskrit word meaning 'gathering' or 'to meet' or a Fair.

The Open Source mela 2009 is happening at IEC, Sun Microsystems, India. Get all the updates at the program blog

Static probes are being gradually integrated into MySQL. As of MySQL 6.0.9, there are around 55 static probes. To use the static probes, you will have to supply an extra option to the configure script, --enable-dtrace. After the build is over, start mysqld. Now open a terminal, and type $pfexec dtrace -l | grep mysql. You should see something like this:

135 mysql23509            mysqld __1cQdispatch_command6FnTenum_server_command_pnDTHD_pcI_b_ command-done
  136 mysql23509            mysqld __1cQdispatch_command6FnTenum_server_command_pnDTHD_pcI_b_ command-start
  137 mysql23509            mysqld __1cQclose_connection6FpnDTHD_Ib_v_ connection-done
  138 mysql23509            mysqld             handle_one_connection connection-start
  139 mysql23509            mysqld __1cMmysql_delete6FpnDTHD_pnKTABLE_LIST_pnEItem_pnLst_sql_list_LXb_b_ delete-done
  140 mysql23509            mysqld __1cHhandlerNha_delete_row6MpkC_i_ delete-row-done
  .
  .

  

As is expected, the function names are mangled. You can view the demangled names with 'dem':

$pfexec dtrace -l | grep mysql | awk '{print $4}' | xargs dem | awk '{print $4}'

which you will show the demangled functions:

handler::ha_external_lock(THD*,int)
handler::ha_external_lock(THD*,int)
handler::ha_external_lock(THD*,int)
handler::ha_external_lock(THD*,int)
handler::ha_external_lock(THD*,int)
handler::ha_external_lock(THD*,int)
mysql_insert(THD*,TABLE_LIST*,List<Item>&,List<List<Item>
handler::ha_write_row(unsigned
handler::ha_write_row(unsigned
select_insert::abort()
select_insert::send_eof()
mysql_execute_command(THD*)
mysql_execute_command(THD*)
multi_delete::send_data(List<Item>&)
mysql_execute_command(THD*)
multi_update::send_data(List<Item>&)
mysql_execute_command(THD*)
.
.
.
handle_select(THD*,LEX*,select_result*,unsigned
handle_select(THD*,LEX*,select_result*,unsigned
multi_update::send_data(List<Item>&)
mysql_update(THD*,TABLE_LIST*,List<Item>&,List<Item>&,Item*,unsigned,st_order*,unsigned
handler::ha_update_row(const
handler::ha_update_row(const
mysql_execute_command(THD*)

I learned and played around a bit with the character sets in MySQL. What you see below is a simple MySQL test script to test the behaviour of MySQL with a couple of indic languages, I know- Bengali and Hindi. The key here is to create a column which will be using the 'utf8' character set and 'utf8_general_ci' collation:

CREATE TABLE t8 (c TEXT character set utf8 collate utf8_general_ci);
--warning WARN_DATA_TRUNCATED 

# Bengali
insert into t8 values(repeat('রবীন্দ্রনাথ ঠাকুর', 10000));

# Hindi
insert into t8 values('शाहरुख़ खान ये कहें कि सुपरस्टार शाहरुख़ खान एक ऐसे अभिनेता हैं जिन्होंने दिल्ली के एक आम शख्स से हिंदी सिनेमा के बेहद ख़ास अभिनेता बनने तक का सफ़र तय किया है.');
insert into t8 values(LOAD_FILE('$MYSQLTEST_VARDIR/hindi.txt'));

SELECT c, length(c) FROM t8; 

drop table t8;

Copy the above script into a file, say ctype_utf8_indic.test and (after copying it to your t/ directory) execute it with the MySQL test runner- mysql-test-run.pl script:perl mysql-test-run.pl ctype_utf8_indic. You should see something like this:

রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্র��াথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ রবীন্দ্রনাথ ঠাকুররবীন্দ্রনাথ ঠাকু  65534
शाहरुख़ खान ये कहें कि सुपरस्टार शाहरुख़ खान एक ऐसे अभिनेता हैं जिन्होंने दिल्ली के एक आम शख्स से हिंदी स बनने तक का सफ़र तय किया है.      831

Of course, the first type you run it, you will have to do a --record, or you may copy my .result file into your r/ directory and see if you get similar results. (My hindi.txt file)

One point to note here is that you won't need to confgiure your OS for either of the languages. However, you will need a editor which supports them. I found 'gedit' handy. I compiled 'vim' with multi-byte support, but haven't had any luck yet.

For some Bengali text, refer: http://www.bbc.co.uk/bengali/ and http://www.bbc.co.uk/hindi/ for Hindi.

Thank you Bar for helping me out!

Just came across this paper:  Optimizing MySQL Database Application Performance with Solaris Dynamic Tracing

Looks useful.

I will resume blogging at my earlier blog: http://amitksaha.blogspot.com
See you there!

I have written about MySQL Query Browser and MySQL Administrator in my article: A graphical way to MySQL mastery on Linux.com

PS: All my writings are archived here at http://amitsaha.in.googlepages.com/writingsandtalks

MySQL Query Analyzer is Sun's latest addition to the MySQL Enterprise tools suite. Here is the press release.

A simple architectural overview of the product is at http://www.mysql.com/common/images/enterprise/queryanalyzeragent_350.jpg