Angelo's Soapbox
Main | Next page »
Wednesday Jun 17, 2009

Diving into the Depths of Drupal with DTrace

I recently presented DTrace at Sun's CommunityONE event. My presentation was focused on observing drupal using DTrace. You can see a replay of the presentation here. You can get the presentation here as well.

If you are an AMP (Apache MySQL PHP) stack user on Solaris/OpenSolaris you can benefit from DTrace. Here is a D-Script that will print the load distribution on your system. Think of it as "AMP-top" if you may! This was tested on OpenSolaris 2009.06

#!/usr/sbin/dtrace -qs

BEGIN
{
	total=mysqlcnt=httpcnt=phpcnt=javacnt=ffxcnt=othercnt=0;
	printf("%10s %10s %10s %10s %10s %10s\n","% MYSQL","% APACHE","% FIREFOX","% PHP","% Java","% OTHER");
}

php*:::request-startup
{
	inphp[pid,tid]=1;
}

php*:::request-shutdown
{
	inphp[pid,tid]=0;
}

profile-1001
{
	total++;
	(execname=="mysqld")?mysqlcnt++:\
	     (execname=="httpd")?(inphp[pid,tid]==1?phpcnt++:httpcnt++):\
	     (execname=="java")?javacnt++:\
	     (execname=="firefox-bin")?ffxcnt++:othercnt++;
}

	

tick-30s
{
	printf("%10s %10s %10s %10s %10s %10s\n","% MYSQL","% APACHE","% FIREFOX","% PHP","% Java","% OTHER");
}

tick-2s
{
	printf("%10d %10d %10d %10d %10d %10d\n",mysqlcnt*100/total,httpcnt*100/total,ffxcnt*100/total,phpcnt*100/total,javacnt*100/total,othercnt*100/total);
	total=mysqlcnt=httpcnt=phpcnt=ffxcnt=javacnt=othercnt=0;

}
I will post other scripts soon. I plan to twitter these scripts as well. Follow me on twitter

Posted at 09:30AM Jun 17, 2009 by angelo in DTrace  |  Comments[1]

Saturday Apr 25, 2009

DTrace presentation in JSconf 2009 Slides

I had an excellent time at JavaScript conference in DC this weekend. Some real smart guys in the room! Very impressive to see how far JavaScript has come! Interestingly many folks knew about DTrace! Here are my slides for the DTrace talk at JSconf 2009

Posted at 02:32PM Apr 25, 2009 by angelo in DTrace  |  Comments[0]

Wednesday Apr 08, 2009

A drupal Aha moment, brought to you by Sun Unified Storage

Have you wondered what exactly an application is doing to your storage subsystem. Now you no longer have to wonder, you can find out exactly what its doing. Recently I was introduced to drupal and for DrupalCon I set up a BOF and demonstrated the use of Sun's Unified storage to observe Drupal. Yesterday I did a screen cast of the demo. You can see it @ "A drupal aha moment, brought to you by Sun's Unified Storage"

You can get one of the Unified storage systems to try out using Sun's Try & Buy Program. Or download a vmware appliance that simulates the Unified Storage.

Posted at 02:23PM Apr 08, 2009 by angelo in DTrace  |  Comments[0]

Monday Jan 12, 2009

Mounting Amazon S3 buckets as a file system on OpenSolaris

Happy New Year.

Recently, I have been trying to make Amazon S3 buckets more "accessible" on my OpenSolaris system. My goal was to "mount" an S3 bucket as a filesystem on OpenSolaris.

I came upon s3fs a filesystem in userspace (FUSE) that uses the Amazon S3 as its data store. Last week I spent a few minutes building s3fs on OpenSolaris.Here are the step by step instructions.

Summary Instructions


Details
  • Building fusefs & libfuse on OpenSolaris
    Install needed pkgs

    Download, build and install Open Solaris version of fusefs and libfuse
    • hg clone ssh://anon@hg.dot.opensolaris.org/hg/fuse/fusefs
    • hg clone ssh://anon@hg.dot.opensolaris.org/hg/fuse/libfuse
    • Read the INSTALL doc for instructions on building and installing fusefs. Here are a few Gotcha's.
      • Make sure to use make and not gmake
      • Add /opt/onbld/bin/i386 to the path
      • cd into fusefs/kernel
      • make
      • make pkg
      • pfexec pkgadd -d packages SUNWfusefs
      • pfexec bootadm update-archive
      • reboot
    • Read the INSTALL doc for instructions on building and installing libfuse.
      • cd libfuse
      • make install
      • make pkg
      • pkgadd -d packages SUNWlibfuse

  • Download and build s3fs

  • Using s3fs
    • First make sure you have an AWS account.
    • Create or reuse an S3 bucket you already have. another-s3-bash is an excellent tool for this purpose. It works well on OpenSolaris. The rest of the instructios assume that you are using another-s3-bash
    • Set up env variables for another-s3-bash
      • export S3_ACCESS_KEY_ID="your access key"
      • echo -n "your secret key" > secret-key.txt
      • export S3_SECRET_ACCESS_KEY=secret-key.txt
    • Create a bucket on S3
      • s3 put fusebucket
    • Put something in the fusebucket
      • echo "Hello S3 World" > /tmp/hello
      • s3 put fusebucket:hello /tmp/hello
    • Mount the S3 bucket
      • s3fs fusebucket /mnt
    • Use it
      • cd /mnt
      • You can do normal filesystem operations like, ls, cp, mv chmod etc
    • s3fs logs everything using syslog. So if you need to see messages look at /var/adm/messages

Hope this helped. Let me know if you have issues. More to come on this front.

Posted at 01:30PM Jan 12, 2009 by angelo in Web2.0  |  Comments[2]

Friday Oct 24, 2008

Optimizing the DTrace logger for MySQL queries

The Data Charmer Giuseppe Maxia did some quick tests on the DTrace script and was wondering if I could optimize it a little.

One issue with the old script is that it prints every SQL statement and this can be pretty expensive. This can be minimized by printing to a file. Here is a script that will do just that. The freopen() is not documented but it opens a file and sends all prints to the file. Giuseppe reports a 30% improvement in logging performance with this improved script. 


#!/usr/sbin/dtrace -qws
BEGIN
{
	freopen("/tmp/sqls");
}

pid$1::*dispatch_command*:entry
{
	printf("%d::%s\n",tid,copyinstr(arg2));
}

One more optimization that you can use with DTrace is the use of aggregates. Aggregates provides summary information. So this script will not provide you with the running log but if performance of the logger is important and you can live with the summary then this is the script for you. 

#!/usr/sbin/dtrace -qs 
pid$1::*dispatch_command*:entry 
{ 
	@[copyinstr(arg2)]=count(); 
}

Strings in DTrace are 256 bytes by default. So these scripts will only show the first 256 chars of the SQL. If you need more then just change the default. Here is the script. 

#!/usr/sbin/dtrace -qws
#pragma D option strsize=1024

BEGIN
{
	freopen("/tmp/sqls");
}

pid$1::*dispatch_command*:entry
{
	printf("%d::%s\n",tid,copyinstr(arg2));
}

Finally, DTrace does the printing/aggregating asynchronously, so the performance of DTrace logging should be better than the typical database logging, at least on a system with enough free cycles.

PS: For those trying out the oracle SQL statements I found a better script than mine here

Posted at 12:20PM Oct 24, 2008 by angelo in DTrace  |  Comments[1]

Friday Oct 17, 2008

Response: DTrace MySQL & SQL statement

A lot of comments on my last blog. Here is my response

How can I get more details for each SQL statement?
The second argument to dispatch_command is a pointer to the THD structure. It contains all sorts of information. You can dig for all the info you may ever need. Only issue is that we are now dealing with the internal structures and these may change for every release. So the script will not be portable. When I get a few minutes I'd blog a sample but your millage may vary.
A better way to get these structures is using the embedded MySQL DTrace probes. These will expose the info you need. For details on the embedded DTrace probes and example scripts on how to get these details see the discussion thread on OpenSolaris DTrace Discuss Finally the tid was printed to give you some amount of isolation based on connections. Not a great solution but good enough in most cases.

Why not just use MySQLProxy?
Because you need to turn it on and restart your mysql instance. With DTrace this process is truly dynamic. But as Kay Röpke explains, in MySQL 5.1 you can turn on query logging on the fly. DTrace of course can be used for many other things, not just query logging. Stay tuned for more to come in my blog

Can I do this for Oracle?
Of course you can. Its just another process as far as Solaris and DTrace are concerned. Here is a simple script that will print the SQL statements for Oracle. The function name is sqlcxt() the third argument is a pointer to the sqlexd structure. Search the web for the definition of the structure for your version of Oracle. Put it in a include file Oracle.h Then copy and run this script


#!/usr/sbin/dtrace -Cs
#include "oracle.h"
pid$1::sqlcxt:entry {

        this->s = (struct sqlexd *) copyin(arg2,sizeof(struct sqlexd));
        self->query = copyinstr((uintptr_t)this->s->stmt);
        printf("%s: %s\n",execname, self->query);
}

Why can't I do this in Linux? Is Sun making Linux the second choice
Kay already provided a very good response. I just want to point one more things. You can use Solaris Branded Zones to run DTrace on a Linux Application. See for details.
Not only are we not treating Linux as a second class citizen we have gone out of the way to help out our friends in the Linux community even though they have been reluctant in getting DTrace ported.

Posted at 09:52AM Oct 17, 2008 by angelo in DTrace  |  Comments[6]

Thursday Oct 16, 2008

Using DTrace to observe the SQL statements on a live running MySQL database

DTrace allows you to instrument any live running application in production without the need of extra coding, application recompile or even an application restart. All you need is that the application is running on an OS that supports DTrace. Today Solaris, OpenSolaris, OS X and FreeBSD are a few that have DTrace built in.

For example here is a D-script that instrument MySQL to observe the SQL statements that is being executed in production. Just a 3 line script like this can be extremely useful to observe a live MySQL database


#!/usr/sbin/dtrace -qs
pid$1::*dispatch_command*:entry
{
    printf("%d::%s\n",tid,copyinstr(arg2));
}
You need to pass the pid of the mysql process as the first argument to this script.

Here is a sample output from the script. We are observing the SQL statements that are executed to bring up the SugarCRM login screen.


# pgrep mysql
418
254
# ./mysql.d 418
11::sugarcrm
11::SET CHARACTER SET utf8
11::SET NAMES 'utf8'
11::SELECT id, name, symbol, conversion_rate FROM currencies WHERE
status = 'Active' and deleted = 0
11::SELECT category, name, value FROM config
11::SELECT id FROM outbound_email WHERE type = 'system'
11::SELECT * FROM outbound_email WHERE id =
'592d612d-3fd2-fa02-f067-48d0467e2da0'
11::
12::sugarcrm
12::SET CHARACTER SET utf8
12::SET NAMES 'utf8'
12::SELECT id, name, symbol, conversion_rate FROM currencies WHERE
status = 'Active' and deleted = 0
12::SELECT category, name, value FROM config
12::SELECT id FROM outbound_email WHERE type = 'system'
12::SELECT * FROM outbound_email WHERE id =
'592d612d-3fd2-fa02-f067-48d0467e2da0'
12::SELECT acl_actions .*, acl_roles_actions.access_override
                   FROM acl_actions
                   LEFT JOIN acl_roles_users ON
acl_roles_users.user_id = '' AND  acl_roles_users.deleted = 0
                   LEFT JOIN acl_roles_actions
12::SELECT count(*) as the_count FROM config WHERE category='info' AND
name='sugar_version' AND value = '5.1.0'
12::SELECT category, name, value FROM config
12::SELECT id FROM outbound_email WHERE type = 'system'
12::SELECT * FROM outbound_email WHERE id =
'592d612d-3fd2-fa02-f067-48d0467e2da0'
12::INSERT INTO tracker
(monitor_id,module_name,date_modified,action,session_id ) VALUES (
'97e1d39b-c036-1e1f-bf2f-48f763a6045c','Users','2008-10-16
15:53:13','login','8fbb9c1a0ac421964fa4a45db4086a46')
12::SELECT user_name FROM users WHERE id='1'
Please note you do not need anything special from MySQL. This will run with any plain vanilla mysql instance with no restart or special flags

How does the script work
We are using the pid provider in DTrace to instrument the dispatch_command method of the live running MySQL instance. This is the method that is called when any SQL statement is executed. The third argument for this method is a string that contains the SQL statement. We are just printing the third argument in the D-script

As always let me know if you need more details

Posted at 11:26AM Oct 16, 2008 by angelo in DTrace  |  Comments[11]

Friday Oct 03, 2008

Using DTrace to change OS version

You may have heard about Solaris' legendary binary compatibility guarantee. Basically we guarantee that any binary that runs on Solaris 7,8 & 9 will run unmodified on Solaris 10.

In spite of this excellent binary compatability we have seen some of our partner's Solaris 9 apps fail to install on Solaris 10 because the installer thinks that "10" is less than "9" (using ASCII compare). I figured that we can use DTrace to fix the version number to allow the installer to work.

So I wrote a simple script that will fix the version of Solaris. Here is the script

#!/usr/sbin/dtrace -qws
	
syscall::uname:entry
/progenyof($1)/
{
	self->addr = arg0;
}

syscall::uname:return
/progenyof($1)/
{
	copyoutstr("5.9", self->addr+(257*2), 257);
}

This script takes the pid of the shell that runs the installer. (echo $$ on the terminal to get the pid) While this script is running, uname -a on that shell will return 5.9 as a the Solaris version.

Posted at 09:46AM Oct 03, 2008 by angelo in DTrace  | 

One Year Free hosting for Glassfish developers


You may have heard that There Ain't No Such Thing As A Free Lunch.[TANSTAAFL]

The Sun's Startup Essentials Program is changing all that with the help of our partner LayeredTech. They are offering one year of free hosting for the glassfish developer community. Click here to apply.

Alex De Marco a happy user of this offer has written up a good getting started guide. Enjoy!

Posted at 09:12AM Oct 03, 2008 by angelo in General  | 

Wednesday Oct 01, 2008

Proud to work for Sun

I have a new role at Sun. I'm a Technical Evangelist in the Sun Startup Essentials team. This is a dedicated team at Sun that helps interesting new startups navigate the complicated world of Web 2.0 technology and funding. If you are a Startup you need to apply to become part of this program. We provide some excellent benefits.

I was at the Hatchery Event in NY last week. I met some wonder folks there and had a great time.

Lynn Brewer is the the real Enron whistleblower. She is the founder of the The Integrity Institute As the name suggest they provide Advanced Corporate Performance Analytics with more than just financials as the basis. She presented her vision during the event and I had an opportunity to talk with her after the event. She mentioned that Sun is among the best in the industry in her integrity rating. Wow! I felt so proud to be part of such a company in these days when we hear of so many scandals.

I also met Jonathan Askin who is the Director of Brooklyn Law School for Incubator & Policy Clinic. He also confirmed that Sun almost always is on the right side on the policy issues. Yes, I do work for a company that not only "Does no evil" but also wants to "Do good" for the right reasons.

Posted at 02:02PM Oct 01, 2008 by angelo in General  | 

Wednesday Sep 17, 2008

Web 2.0 Expo NY

Hello from the demo floor of the Web 2.0 Expo in New York

It has been a lot of fun getting to meet with a lot of Web 2.0 companies. It blows my mind to just see how much this market has matured.

If you are at the Web 2.0 Expo NY please stop at the Sun booth. Tell them Angelo sent you and get a nice OpenSolaris cap or T-shirt.

Come by my BOF on Observing the Web 2.0 Application in Production September 9/18/08 Thursday 7 PM at "Boardroom C-Hudson Hotel". We will have an interactive session. So would love to hear your ideas and feeback.

Also check out Project Kenai and Zembly at the Sun booth.

If you are a startup. The SSE program is just the thing for you. See Sun Startup Essentials

Come by booth 1701

Posted at 12:14PM Sep 17, 2008 by angelo in DTrace  | 

Wednesday Aug 27, 2008

Engineering Webcast: OpenSolaris, A Boundless Development System

Time for some shameless plug. I host a engineering webcast where I get to talk with some pretty cool folks. We just released one more of these webcast. This one talks about OpenSolaris. If you are a developer you need to check this one out. Both Hugo and Marty are exciting speakers. In this talk we some details on both the technical and business side of OpenSolaris. Enjoy

http://www.snpnet.com/customer_pub/sun/ISV_OpenSol/

So what else would you like to hear about!

Posted at 04:09PM Aug 27, 2008 by angelo in Web2.0  | 

Tuesday Jul 15, 2008

DTrace screencast

Its been a while since I blogged here. So wanted to do blog some good content. I work in the Open Source team at Sun and I'm focusing on Emerging Markets. Its a pretty exciting work and I'm enjoying very bit of it.

Recently I came across an open source project called the "Jing Project". Jing makes creating screen casts a breeze. I've been playing with it and created a few screen casts to demonstrate DTrace. If you have not seen DTrace in action here is your chance.

To make it easy for download and viewing the screen cast has been split into 5 parts. Easy on the bandwidth and easy on the eyes. So without much more ado here are the screen casts. Enjoy and provide feedback.

DTrace screen casts
Name Description
Introductory Presentation 01-Intro.swf
Observing System calls 02-demo-syscall1.swf
System call arguments 03-demo-syscall2.swf
Library calls and arguments 04-demo-libc.swf
DTrace and JavaScript 05-demo-javascript.swf


You may need a flash player to view these screen casts.

I plan to do DTrace tutorials in the next series of screen casts. Enjoy and don't forget to provide feedback.

Posted at 10:22PM Jul 15, 2008 by angelo in DTrace  |  Comments[3]

Tuesday May 22, 2007

DTrace meets the AMP (Apache MySQL PHP) stack

During JavaOne this year Rags and I did a hands on lab on DTrace. The topic was "Using DTrace on Java and other Web 2.0 Languages in Solaris". Due to some unforeseen reason the lab was scheduled from 8:45 PM to 10:45 PM (Yes PM) and that to on the day of the big party at JavaOne. So I was expecting to be speaking to an empty room but lo and behold we had close to 100 engineers ( a few pretty drunk) and most stayed until the finished the lab.It was pretty heartening and was worth all the efforts that went into it.

One of the highlights of the lab was a script that we developed. It lists the JavaScript & PHP function entry and MySQL SQLs that happen live system. You need the following prerequisite for the script to be fully functional.

Here is the script. It takes one argument - the pid of the mysqld process in your system.

#!/usr/sbin/dtrace -ZqFs
/* We use terminal escape sequence to change color
 * Javascript - Red
 * PHP - Blue
 * MYSQL - Green
 */
 
trace_mozilla*:::js_function-entry
{
        printf("\033[01;31m%s\033[0m\n",copyinstr(arg2));
}

trace_mozilla*:::js_function-return
{
        printf("\033[01;31m%s\033[0m\n",copyinstr(arg2));
}

php*:::function-entry
/arg0/
{
        printf("\033[01;34m%s\033[0m\n",copyinstr(arg0));
}

php*:::function-return
/arg0/
{
        printf("\033[01;34m%s\033[0m\n",copyinstr(arg0));
}

pid$1::_Z16dispatch_command19enum_server_commandP3THDPcj:*
{
        self->sql = copyinstr(arg2);
        printf("\033[01;32m%s\033[0m\n",self->sql);
}

 You will see a nice color coded output. Javascript calls in Red, PHP in Blue and MySQL in green.

In subsequent blogs I try and go through the process of developing this code. Meanwhile enjoy Solaris DTrace

Posted at 01:17PM May 22, 2007 by angelo in DTrace  |  Comments[1]

I met Vinod Khosla

During JavaOne this year Rags and I went to a nice South Indian restaurant, Dosa, in San Francisco. This is an excellent place by the way and I highly recommend it. I was looking around and it seemed like we were the only PIO (Persons on Indian Origin) around until I spotted a family sitting just behind us. On closer observation it looked like someone familiar. I checked twice and quitely told Rags, "Hey Vinod Khosla is sitting behind us?"

We debated for a few minutes if it was right to intrude but better wisdom prevailed and we walked up to him and actually shook his hands.He was pretty gracious and freely chatted with us. This was definitely one of the highlights of JavaOne for me.

For those of you who do not know Vinod, he is the founder of Sun He is also the founder of Daisy Systems and Khosla Ventures and help start companies like Juniper networks and Excite etc. See his WikiPedia Entry for more details.

Posted at 12:03PM May 22, 2007 by angelo in General  | 

Archives
« June 2009
SunMonTueWedThuFriSat
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today


Links



Referrers

Today's Page Hits: 62