Mani's blog: Mani Chandrasekaran's Weblog

I have been a user of twitter for around a year now, and I love it. I am not yet a big follower of people, but I hope to gradually follow more people soon. I came across this interesting chart from comscore.com, posted by Sarah on the phenomenal, exponential growth of twitter traffic in the US and around the world. You need to see it, to believe it .

Image linked from from: http://www.comscore.com

These kinds of growth, are very interesting to me, both from my day job as an Enterprise Architect, as well as an observer of the technological trends. The next question, was how is twitter bulit, what kind of technology does it use. highscalability.com gives some idea about the technology stack -- MySQL (MySQL handled 2,400 requests per second, Only one master and one slave), Rails, Memcached .., all running on Sun servers

There is also an article at artima.com with Bill Venners, where its mentioned that they have switched on to using Scala for some of the back-end stuff and why they did it . Scala ?? I did not know, anything about Scala until now. But as per http://www.scala-lang.org, "Scala is a general purpose programming language designed to express common programming patterns in a concise, elegant, and type-safe way" . Yet another language, but I guess, the motto is to use what makes sense and works !!

We were at a potential customer last week, where the problem statement was:

1. We have several applications using RDBMS as the user repository, for user authentication. There are several databases for users, which may potentially have duplicates/li>
2. We are expecting to grow rapidly, the current requirement of users is large and expected to grow exponentially.
3. We would like to use Single sign on, in future

Considering the number of users required, and the primary requirement being user authentication, using an LDAP server like Sun Directory server enterprise edition (DSEE), seemed to make sense, and that's what we suggested. The next question was (similar to other customers), why do we need LDAP? and how does it compare it to a RDBMS? There are several, several reasons for using an LDAP server (like Sun DSEE or OpenDS) in these situations, like read's tend to be a lot faster, high availability situations like multi-master replication etc. Thanks to my friend Rajiv, we managed to get hold a fantastic technical white paper that highlights the difference's between LDAP and RDBMS, and where to use, what. I have uploaded it here, as I was unable to refer to the original Sun location.

I have reproduced this table here, from the whitepaper :

Of course, this does not solve the problem of removing user data duplication, which was the other requirement. For that, you will need a product like Sun Identity Manager, which will be used for data reconciliation, user synchronization, to basically create a "authoritative user repository" !! This is the key step, the next step, will be to roll-out things like single sign on etc ..

Bangalore best Indian city to live in, says Mercer survey April 28, 2009

As per rediff.com, "Bangalore has emerged as the best Indian city among New Delhi, Mumbai and Chennai in terms of better quality of living for expatriates, according to a latest worldwide survey of cities by global HR consultancy Mercer.

According to Mercer survey, '2009 Quality of Living global city rankings' for 215 cities, Bangalore has topped the list among Indian cities, while the country's financial hub Mumbai has witnessed a drop in rankings this year mainly due to a decline in stability and security conditions.

For a hard-core Bangalorean like me, this is a reaffirmation of Bangalore's lovely weather, friendly people, mix of Indian and western culture, cosmopolitan nature, that appeals to a lot of Indians and others ...

As with a lot of people, I am a user of the web2.0 sites like LinkedIn, Twitter, gmail, Yahoo etc. Infact sites like LinkedIn have become really addictive, and I do check the website atleast a couple of times a day, if I have time and am not on the road. So, on a Sunday morning, I opened Linkedin.com at around 10:15 am Indian standard time and I got this message :

I was a little annoyed, that my favorite website is undergoing maintenance, and more so during day time in India !! Of course, it got me thinking, about the larger aspects of this issue. In this day and age, when your business is used by people across geographies, do things like having a maintenance window during offpeak hours in your local region (in this case the US), really work? Is it acceptable, and any ideas to overcome this?

A google search on "Linkedin architecture" came across this piece of information about LinkedIn's architecture. A couple of points catch my eye, "The Cloud is a server that caches the entire LinkedIn network graph in memory", "Rebuilding an instance of The Cloud from disk takes 8 hours" .. The components used in the architecture like Tomcat, Jetty, MySQL, Oracle etc are certainly capable of 99.999% availability, if architected that way. The Linkedin architecture, is great stuff to learn and understand, how high volume websites are built. There are plenty of details on how caching is done, LinkedIn also uses a push based architecture for generating content (could this be the reason???). I cannot really draw conclusions, on how to avoid this.

But, I have a couple of points, on how downtime can be avoided, and at the same time, allocate time for upgrades (hardware and software) and maintenance:
1. Use a Content Delivery Network (CDN) like Akamai, to deliver Content. This way, you have websites which deliver content to different regions across the globe, without downtime in one region affecting others. Of course the cost, could be factor. The other thing to consider would be, if the contents are very dynamic, how will this work?
2. Use a Rolling Upgrade kind of strategy. That is, if you have several servers in a cluster, high availability setup, remove one server from the cluster, upgrade and push it backup to the cluster, and then take the next one. Of course, there may be a few minutes/hours, when the versions of the applications. will be different. But, you can avoid downtime.
My thoughts, on a Monday afternoon Hope this makes sense !!!

MySQL has been having a lot of traction with customers, partners and developers in India. A very popular architecture for using MySQL for large scale deployments is the MySQL Master/Slave replication (Replication enables data from one MySQL database server (called the master) to be replicated to one or more MySQL database servers (slaves)).
Credit: Image is linked from http://dev.mysql.com/doc/refman/5.0/en/replication-solutions-scaleout.html.

One of things that change for developers, who are used to developing with other databases like Oracle, MS SQL server is the question on connecting to a MySQL master/Slave setup. There are several good resources which explain how to do this:
1. From MySQL web site: A snippet of the code from that article is reproduced below :

import java.sql.Connection;
import java.sql.ResultSet;
import java.util.Properties;

import com.mysql.jdbc.ReplicationDriver;

public class ReplicationDriverDemo {

  public static void main(String[] args) throws Exception {
    ReplicationDriver driver = new ReplicationDriver();

    Properties props = new Properties();

    // We want this for failover on the slaves
    props.put("autoReconnect", "true");

    // We want to load balance between the slaves
    props.put("roundRobinLoadBalance", "true");

    props.put("user", "foo");
    props.put("password", "bar");

    //
    // Looks like a normal MySQL JDBC url, with a
    // comma-separated list of hosts, the first 
    // being the 'master', the rest being any number
    // of slaves that the driver will load balance against
    //

    Connection conn =
        driver.connect("jdbc:mysql://master,slave1,slave2,slave3/test",
            props);

    //
    // Perform read/write work on the master
    // by setting the read-only flag to "false"
    //

    conn.setReadOnly(false);
    conn.setAutoCommit(false);
    conn.createStatement().executeUpdate("UPDATE some_table ....");
    conn.commit();

    //
    // Now, do a query from a slave, the driver automatically picks one
    // from the list
    //

    conn.setReadOnly(true);

    ResultSet rs = 
      conn.createStatement().executeQuery("SELECT a,b FROM alt_table");

     .......
  }
}

Note: Please refer to the mySQL web site for any updates.
2. If you are using Spring, Hibernate and other popular frameworks, an article at http://neilhan.blogspot.com/2006/11/spring-hibernate-and-mysql-replication.html tells you how to do it.

The Sun Partner Advantage Program with Open Access Channel Program and Software Specialties is being rolled out in India. Now it's easier than ever for resellers and Sun partners to offer the full range of Sun Microsystem's software to their customers. The Sun Partner Advantage Open Access Channel Program allows resellers to sell Sun software without entry barriers, and the Software Specialties Program provides partners with a simple framework for building profitable services practices, such as Service Oriented Architecture (SOA), Glassfish, Identity Management and MySQL, utilizing Sun software.

We are kicking off this program with a two day Sales person focused event and a two week long technical bootcamp. If you are a partner in India, and would like to take advantage of this, let me know (email me at mani-dot-chandra-at-sun-dot-com). Seats are limited !!

One of the things that keep happening in the life of a person in Software sales and technical pre-sales(like myself), is answering the RFP's for various customers. This is specially true for Government and Public sector companies and also in some of the larger private sector projects. I will be compiling a list of handy blogs for the various Sun software products and how/where to look for various information when answering RFP's.

There are three IMPORTANT things that are part of the RFP process :

1. RFP - Answering the RFP. This is the most important part, as in a lot of situations, a specific vendor will have influenced the RFP. Its important to highlight to the customer, why those RFP questions are biased, and ask for changes. For example, there were RFP's for application servers, which specified that application server's should comply to .Net and ECMA standards. Obviously, this is totally biased against the Java/J2EE based application servers. One of things that I have learnt the hard way, is to ensure that the matrix we fill in, with the RFP responses, should also have links to actual documentation or articles or comments. Fortunately for Sun, there is a ton of information at docs.sun.com, bigadmin, blogs.sun.com and developers.sun.com
2. Solution architecture document : The key thing in the solution architecture, will be any third party software components and the sizing to fit in with their non-functional requirements. We do have access to some Sun internal data, but a ton of information exists in the internet as sizing guides and also in the various benchmarks published externally. The sizing should meet the key availability requirements specified by the RFP. If third party software is required, we need to do due diligence about the requirements of the third party software.
3. References - Most customers ask for references. We do have a lot of publicly referenceable customers available at http://www.sun.com/customers plus a a very large customer list that can be quoted in a private fashion.

We, in India, have been having some terrific wins recently by positioning Sun software, in some very large deals (both revenue and in size/complexity). Sun software being largely open source and always standards based, has been a great combination in these difficult economic times
I will be touching upon solutions, Enterprise architectures and responding to RFPs for Glassfish app server, Glassfish web space portal, Identity and Communication suite's in subsequent posts.

Advertisements spotted in Bangalore, Enjoy

I had enrolled in an Executive General Management Programme (EGMP) at Indian Institute of Management, Bangalore last year in May 2008. I successfully completed the course last week, and was awarded the certificate.

The course was great, and the courses that I especially liked were Macroeconomics (Prof Shamal Roy), Financial accounting (Prof Padmini Srinivasan), Microeconomics (Prof Ranganathan), Corporate Strategy (Prof. Rishikesha Krishnan). The professors were generally very good, with a wealth of experience and the great atmosphere at IIMB was a added advantage. The books that were given as course material was generally outstanding !!

A few snaps from IIMB:


Apart from Bangalore, there were also off-campus students viewing through video conference from Chennai, Hyderabad, Pune, Delhi and Mumbai. I made a lot of great friends, and my batch mates for ten months are some of my best buddies now. I met people from different industries, Manufacturing, Pharma, etc and it was a microcosm of the industrial scene in India. I will definitely recommend this to any person who is in the management stream and wants to brush up on management topics. A person who also deserves great mention is Mr Vedi, program assistant, who worked like clockwork and made sure the programme ran very well. Special mention about the great IIMB food, also requires mentioning, I never knew vegetarian food tastes this good

Cxotoday.com has got a news article on the Sun IDM event done with Deloitte.

Sun and Deloitte are organizing a roadshow at Delhi and Mumbai on Identity compliance. The details of the roadshow are:

Achieving Compliance and Efficiency - through Identity & Access Audits

The recent proliferation of Fraud, Regulatory and Compliance requirements have increased the cost of compliance, created audit fatigue, and taken valuable cycles away from risk management, compliance, information technology & lines of businesses.

Every organization faces the need to have defined, effective and efficient processes to manage against Fraudulent or Unauthorized Access to their critical business assets and information, by • Granting Right Access to Right People “in-time” • Changing Access “in-time” when users exit the organization, or change role & responsibilities • Performing periodic review of ‘Who has access to what’ • Performing periodic review of Segregation of Duties

To address these needs, Deloitte and Sun have developed a business aligned approach and methodology, which integrates leading Identity and Access Management practices with industry leading IAM technology. Our solution will enhance the quality, automation and efficiency of your access audit, compliance and attestation processes.

We are pleased to invite you to an exclusive invitation only event in which we will present our Point-of-View on how you can Manage Access and meet your Audit & Compliance requirements and secure your critical business assets and information.

Venue Delhi : Hyatt Regency
Date: 17th March 2009
Time: 6:00 pm - 8:30 pm (Followed by cocktail & dinner)

Venue: Mumbai Hyatt Regency Date: 19th March 2009
Time: 6:00 pm - 8:30 pm (Followed by cocktail & dinner)

If you would like to attend, and you are at a fairly high level in the org hierarchy to influence decisions , please drop in a email at mani-dot-chandra-at-sun-dot-com.

The CNN rankings for the World's Most Admired Companies for 2009 is out. Sun has moved up, and now ranks 5th in the list. The complete list is at http://money.cnn.com/magazines/fortune/mostadmired/2009/industries/10.html.

I had to blog about this. Verizon is live with 40 million users, 1 million logins per day, and peaks at 4,000 logins per minute, using OpenSSO and Sun Directory server !! As an architect, this is the kind of scalability that we like to brag about (publicly!!) . Please see the blogs by Dan Raskin and Nick Wooler. The Verizon presentation is at http://blogs.sun.com/raskin/resource/Verizon_OpenSSO_Gartner_Preso_Nov08.pdf .

A little belated, but anyway

• Launch of the Glassfish portfolio : The Web stack based on Glassfish, is complete and has been launched. The portfolio consists of :
  1. GlassFish Enterprise Server: The fantastic Java EE compliant app server.
  2. GlassFish ESB: Lightweight and agile ESB platform, I have a comparison on the features between Java CAPS and ESB in one of my posts below.
  3. GlassFish Web Stack : is a complete LAMP/SAMP cross-platform portfolio of Web-tier technologies. Plugnplay, as per the customer needs.
  4. GlassFish Web Space Server: Our Portal service offering, in a new avatar. I love this, have been playing with this, since last Java One. Our customers in India, like the look and feel of this product.
• Glassfish Enterprise manager - Similar to the MySQL model of advisors, we have released three great add-ons, for SNMP monitoring, Performance Advisor and Performance monitoring. These add-ons, come with the paid Glassfish enterprise server, hence more reasons to get a supported version, if you are using the free Glassfish server. The add-ons are available for download from sunsolve.sun.com, for customers with valid contracts. The Glassfish Aquarium has great blogs and articles on this release.
• Sun web space server - The new portal server based on Liferay and the previous Sun portal server has been launched. The docs are available at http://docs.sun.com/app/docs/coll/1863.3?l=en and also the portal blog is alive with lots of posts and articles. Compliance to JSR 168, JSR 286, JSR 170, and WSRP 2.0, having an integrated Content management system, lots of web 2.0 gadgets, ease of installation/monitoring/administration, easy fitment with OpenSSO, TCO being highly competitive, the only "real" enterprise class open source portal server, are some of the great features, that I can see resonating with customers in India.

Our boss, Scott's column on Open Source and its relevance to India and the world , appeared in today's Livemint.com (a tabloid style business news paper, in a JV with WSJ) . Excerpts from the website:

Powering India’s economic growth requires a mix of software rights solutions, not proprietary ones alone - Scott McNealy

Today, nearly every corner of the world faces the challenge of a stagnant or shrinking economy. Bleak economic forecasts, shrinking budgets and increasing pressure on businesses and governments to meet the needs of their customers and constituents—often with less resources to do so—are becoming commonplace. While I’m not naïve enough to suggest a “one-size-fits-all” cure for these problems or that the solutions will be driven by only one industry or region, I do believe that, because technology and innovation drive global economic progress, the remedy for many of these challenges is in our hands.

India is, of course, not immune to these challenges. But the country is very well positioned to meet them. India is one of the world’s fastest growing tech economies and one of the leading participants in the global shift towards free and open source technologies—those eschewing the dependencies of cost and barriers to access that often “come standard” with proprietary technologies. As such, I believe India can play a central role in fostering and adopting the innovations driving its own economic and social growth as well as positioning itself for a larger role on the global economic and technological stage.

India’s use of open source technology and its part in the development and deployment of open standards is not new. Sun Microsystems estimates more than three-quarters of a million Indian developers are members of the Sun Developer Network, actively contributing to communities built around MySQL, OpenSolaris, OpenOffice.org and Java. Indian companies such as Life Insurance Corp. of India, Axis Bank, Canara Bank and Tata Communications use open source technologies as a core part of their business. State governments are also embracing open source. Kerala took the lead in open source when it became the first state in the country to completely banish proprietary software in the mandatory IT test administered to half a million students every year. Even the voting systems for popular TV shows such as Kaun Banega Crorepati and Indian Idol run on open technology. In fact, a recent report from the India Institute of Management, Ahmedabad, seeking to quantify the economic impact of open source Java in India, estimates that the value of the “Java economy” in India is approximately 2.1% of the Indian GDP.

More at http://www.livemint.com/2009/02/17231746/India8217s-open-source-futu.html.