What happens downstream?: Scott Mattoon's blog

Registration for DrupalCon Paris opened this weekend, and already several sponsors have claimed coveted top sponsorship spots for the September 1-5, 2009. Whether or not Sun will sponsor for the 5th time in a row is something I'll be checking into over the coming weeks.

Although the agenda and session schedule for DrupalCon Paris has not yet been posted, there is already some interesting content planned for the event, including full day immersion style training in Commercial Training tracks. Whether or not Sun sponsors this DrupalCon, I plan to make this one my 5th in a row.

In other DrupalCon news, San Francisco is making a bid for North America's DrupalCon in March 2010.

Organizers of DrupalCon DC,  in the lead up to the event, asked Sun and the other event sponsors a few questions about their relationship with Drupal.  The first question was:

How does Sun work with Drupal?

I provided this answer:

There are too many ways in which Sun works with Drupal to list them all here, but some of the highlights are:

• In 2005 Sun donated a server to drupal.org when scaling and performance problems were hampering growth
• Then in 2007 Sun donated another server to further propel the scale up of d.o
  
• Sun has used Drupal in building several important online communities:
• TED Prize winner Architecture for Humanity's Open Architecture Network
• Freshbrain - a technology exploration platform for youth
  
• the Open Office extensions site
  
• the Sun Learning eXchange
  
• Plus many organizations run Drupal on Sun technology
• Employees at Sun have been using Drupal internally for lots of things, including:
• as representative AMP stack environment and workload for lots of benchmarking and performance testing
• to demonstrate certain technologies, both in Sun internal training, and with Sun's customers, e.g., using the Webstack pre-built bundle of AMP and other open source packages for OpenSolaris, Virtualization with Solaris zones, MySQL installation and tuning, and PHP code analysis with DTrace.
  
• Sun has created a NetBeans plugin wizard for developing Drupal modules
• Sun is currently investigating system appliances for data archive and CMS that would bundle Drupal
• Of course, Sun is a large contributor to some of Drupal's most critical underlying open source technologies:
• MySQL AB is a Sun company
• many PostgreSQL core contributors are employed at Sun
  
• many PHP core contributors are employed at Sun
• Sun has contributed more FLOSS code than any other single institution*. (Though, we think Solaris is technically a great choice for Drupal.)
• and let's not forget that Sun VP and inventor of Java, James Gosling, defended Drupal inventor Dries Buytart in his PhD examination at Ghent.
  

Clearly, Sun is deeply connected to the Drupal community in many ways.

All the Q&A from DrupalCon's sponsors will be posted here on the DrupalConDC site.

---

* Entry corrected to say Sun has contributed more FLOSS code than any other single institution, not Sun has contributed more code to the Linux kernel than any other single institution (although I think I did read that somewhere, it's not substantiated in this paper).  Thanks to Matt for point that out - it's major difference.

No sooner had we put the wrap on an April 9 Commonwealth Club panel interview on Collaborating for Change, than PBS announced a really cool collaborative project on Nova to design the "Car of the Future".   Both of these recent productions focus on the application of open source design to social and economic needs beyond software.  The promise of open source economics is popping up everywhere.  It must be something in the water, (or the atmosphere).   Network based open source design efforts have been written about before, and there's more than a few established non-software open source design projects, but they were hardly regarded as mainstream.  And open source as a business model has been a fringe enterprise.  But all that is changing.

The upcoming Nova special, and the Commonwealth Club interview (with Amy Novogratz, Kate Stohr, Maria Giudice, and myself (video courtesy fora.tv)) serve as proof points that this phenomena has exceeded meme status and is spilling over into the broader socioeconomic graph.   But we knew this was inevitable, right?  We just needed the right conditions for humanity's collaborative tendency to come out of the proprietary deep freeze.

The substrate upon which this new culture is rising pairs flexible licensing models a'la Creative Commons with accessible technology for building collaborative online communities a'la Drupal and Wordpress,  Yahoo!groups and PBWiki.  Among the catalysts for this reaction are frustration over obscene economic inequities around the world, abuses of people and planet for profit, and utter neglect by federal governments.  As was discussed here in the video interview about the Open Architecture Network, these frustrations can be overcome by collaborating for change on the net.

Need more proof of the trend toward an open source economy?  Just check with the folks at Open Everything.  They're tracking numerous open collaboratives, which are exogenous to  the software world, but infused with many of the same principles, practices and tools as open source software projects. 

One of the most prominent tools applied to these new collaboratives is Drupal, and we discuss it's role in the Open Architecture Network in the video (at :37:30, :46:30, and :51:00).

Ten years ago who would have imagined that:

• IBM would begin opening their patent portfolio?
• Major news bureaus would sponsor citizen journalism?
• Educational curriculum would be opened and shared for free?
• The Pirate party could get on a national election ballot?
• Sun Microsystems would convert its software portfolio to open source, and it would be profitable?  (Well, that one you might have imagined.)
  

Yet these, and plenty of other examples show that collaborative culture is on the rise.  Does this signal the next generation economy in which businesses profit less from market lockout and legal protection and more from direct value delivered in open markets?  Or does it lead to a more fundamental shift wherein socioeconomic prosperity derives less through commerce than through collaborations for which the primary incentive to contribute is sociocentric good? 

In his keynote at DrupalCon Boston on March 5 Brian Aker discussed the latest approaches to six problems related to scaling up MySQL:

1. Caching
2. Partitioning
3. Replication
4. Batch Processing
5. Understanding Performance
6. Routing

I had a chance to sit down with Brian after his keynote to ask him questions about a few of these approaches.  Here is the interview:

Brian Aker on Scaling MySQL

Thanks to Vic Germani for producing this audio.

---

 Resources:

• Brian's blog
• memcached
• MySQL 5.1 Partitioning
• DRBD
  

Back now from DrupalCon, I'm parsing all that happened last week in Boston.  For me it was a whirlwind, interrupted by a plethora of hassles, including a nasty head cold, keyboard and trackpad on my MBP crapping out, a crashed demo, and several hours separated from my Treo while it rode around in the back of a Boston cab.  All that negative energy converging on me was more than offset by the positive vibe at the four day conference.  The kindness of the cabbie who drove crosstown to return my phone helped too.

One of the highlights for sure was spending time with a new Sun colleague, Brian Aker from MySQL.  We had breakfast at Henrietta's near Harvard Square before his keynote on Wednesday.   I asked him about the merger with Sun, what's next for MySQL, and how he'd like to see our field organizations work together.   He said the merger has been pretty well received and there was a general appreciation at MySQL for Sun's commitment to open source (something I hope will rub off on Brian's Slashdot amigo Chris Dibona, who conspicuously left Sun off of his Tuesday keynote list of companies that "get" open source).  There is a tradition of collaboration between Sun and MySQL too, which Brian indicated ought to help smooth the integration.   Lot's of his work is going into memcached these days, particularly in the libmemcached client.  He cleared up a misconception for me regarding Innodb: since Innodb is GPL'd, the risk of Oracle smothering it is nil - the community is driving it, and it's not the dead end many had feared.   What's next?  Don't expect to see MySQL 5.1 until 2009; do expect a maturing and further specializing application of the MySQL engines MyISAM, Innodb, BDB, and Archive; and plan for an adoption ramp for DRBD.  Brian had some great advice for Sun's field engineers: get familiar with MySQL technology by taking advantage of the many training resource available at MySQL.com.  MySQL University is a great place to start, (be sure to catch Brian's talk on EC2 March 29).  I also caught some good audio one-on-one with Brian after his keynote which I will post separately, along with his advice on scaling up your database.

RDF and Semantic Web were topics of much conversation and at least one BoF session.  With the addition of RDF modules in Drupal 6, developers can mashup data from multiple sites in very interesting ways.  If Web3.0 is massively distributed data mining, indexing, and mashing it all up, then Drupal is positioned to be the portal for this convergence, as Dries Buytaert resolutely declared in his Monday keynote

I gave a talk on running Drupal on Sun, with some help from Chris Cheetham from Project Caroline, at the end of the day on Wednesday (slides at right).  As luck would have it, my demo froze up, but I did manage to show Drupal running in a Solaris Zone, and DTrace to count function calls from Drupal.  Chris's demo of Drupal deployment to Project Caroline went much smoother.

Another highlight was awarding the grandprize Sun Fire T1000 server to the winners of the Showcase Site competition.  PingVision won it for their work on Popular Science Magazine.  Congratulations to Kevin Bridges and the rest of the crew at PingVision.

There was a lot of support for the next DrupalCon to be held in Hungary this fall.  It will be hard to top the Boston event, but I know this community will do their best to have the best one yet.

---

• DrupalCon Boston 2008 on flickr
• Recordings of DrupalCon Boston 2008 sessions
  

Sitting in the Is Beauty Truth? session at TED2008 today, I am reminded of a phone conference with TED Curator Chris Anderson last fall to bring him up to date on the status of one of the previous year's TED wishes.

When Sun wrapped up its formal role in developing the Open Architecture Network (OAN) it handed over a sustaining challenge to the site's owner and community leader, Architecture for Humanity (AFH).   When TED Curator Chris Anderson asked Sun why the TED Prize winner was left in a lurch I gave a short answer, "It was primarily due to reasons of expediency".  In actual fact, Sun never walked away from AFH.  Sun was, and continues to be, committed to their success and continues to be involved.  As of today, we now we see a clear path to a sustaining model that leverages the Drupal community and frees AFH from the dependence cycle it was caught in with Sun.  I look forward to bringing that good news to Chris before the conference wraps up on Saturday.

The first step on this path is to refactor the site such that it runs on an unadulterated Drupal core.  To do that AFH and Sun have contracted with CivicActions to migrate the OAN from a hacked Drupal 4.7 to a clean Drupal 5.X.  (It was the hacking aspect that I explained away to Chris Anderson as "expediency".  Corners were cut, compromises were made, but AFH's and TED's primary goal, to launch the site at TED2007, was achieved.  Incidentally, of the three TED2006 prize winner, only AFH's wish was realized by TED2007.)  CivicActions won the bid to perform the migration by doing a professional and efficient assessment of the OAN's current state and the effort required to bring it up to the high standards of a showcase Drupal site.

My next few posts will describe the process of setting up this development environment as we open Chapter 2 in the OAN's odyssey.  I'll describe how we use OpenSolaris to enable efficient development, testing, and deployment for multiple contributors working on multiple tasks and timelines.

For more on why OpenSolaris was chosen as the development and deployment platform for the OAN, see this article on the Sun Developer Network, and this brief interview.

Whilst installing Drupal 6.0 on Solaris Express Developer Edition (SXDE) 1/08 I ran into a few glitches with the brand new Webstack, which makes it's debut in this build of OpenSolaris.  (SXDE is Sun's distro of OpenSolaris.  It's the best way to get access to all the latest stuff in a relatively feature complete distro of OpenSolaris without having to build the whole O/S yourself.)  With the advent of Webstack integration you don't need a separate download to get all the AMP stack integration and optimizations previously only available in Coolstack.

The executive summary of the solution to the Drupal 6.0 install glitches is:

1. Edit  /etc/php5/5.2.4/php.ini  to add '.' and Drupal's base dir (/opt/drupal-6.0, in my case) to PHP's include_path:

   include_path = ".:/usr/php5/5.2.4/include/php:/opt/drupal-6.0"  

2. Spoof PHP into thinking it's using an older MySQL client:
   

   ln -sf /usr/mysql/5.0/lib/mysql/libmysqlclient.so.15 \ 
   /usr/mysql/5.0/lib/mysql/libmysqlclient.so.12 

Now, with these fixes in place, I have the advantages of the Service Management Framework (SMF) and DTrace, plus an AMP stack compiled with optimizations for Solaris and the processor architecture (AMD64, in my case). 

If you're interested in a more detailed account of the glitches and fixes, read on...

---

The first glitch prevented the Drupal index.php page from rendering, and appeared in the apache error_log as:

PHP Warning:  include_once() [<a href='function.include'>function.include</a>]: Failed opening 
'includes/install.inc' for inclusion (include_path='/usr/php5/5.2.4/include/php' 

which was remedied by adding '.' to the include path in include_path in /etc/php5/5.2.4/php.ini

include_path = ".:/usr/php5/5.2.4/include/php" 

Then, proceeding to the database setup, MySQL gave an error:

"Client does not support authentication protocol requested"

This was easily resolved by the procedure posted on http://dev.mysql.com/doc/refman/5.0/en/old-client.html, but that's a compromise on MySQL password strength, so not ideal.   This allowed me to proceed to the next glitch, which apache error_log explained as:

"ld.so.1: httpd: fatal: relocation error: file 
/usr/php5/5.2.4/modules/mysqli.so: symbol 
mysql_set_local_infile_handler: referenced symbol not found" 

After much hair pulling, Sriram pointed me to the solution he worked out for the same problem with MediaWiki installation:

ln -sf /usr/mysql/5.0/lib/mysql/libmysqlclient.so.15 \
/usr/mysql/5.0/lib/mysql/libmysqlclient.so.12  

and another addition to include_path in /etc/php5/5.2.4/php.ini to add Drupal's base dir

include_path = ".:/usr/php5/5.2.4/include/php:/opt/drupal-6.0" 

Turns out that spoofing the library name like this also solved the "Client does not support authentication protocol ..." problem too, so I'm back to full MySQL password strength.

After these three simple but obscure fixes it was all clean sailing.  I now have Drupal running in six zones across two separate instances of SXDE 1/08 using the latest Webstack. 

Last week Sun Developer Network posted a video interview with me that was taped way back in early October, in which Kuldip Oberoi asked me about Drupal.  A lot has changed for Drupal in the ensuing four months, so a quick update is called for:

• Drupal core downloads have reached a rate of 100,000 per month, which puts the total download count at about 1 million
• Drupal 6.0 was released February 13
  
• There are 573 registered attendees and 172 proposed sessions for DrupalCon Boston, March 3-6, and still counting.
  
• Several high profile Drupal sites have received a lot of attention in the media, including Warner Brothers Records, Amnesty International, SonyBMG, Harvard University, and the Open Architecture Network.
• Drupal creator, Dries Buytaert, has formed a company, Acquia, which, as Ubuntu is to Linux, Acquia is set to be to Drupal
  
• Speaking of usability, my alma mater, University of Minnesota, has partnered with Dries and Drupal to do usability testing.
• Lots of headway is being made bringing Drupal to the Semantic Web, with an active group focused on RDF integration.
  

And that's just a sample of the news emanating from Drupal in recent months.  Clearly, this community is not waiting for the future to come to Drupal, they're creating it with Drupal.

 

I'd never seen the reference outside of Sun before today, but there it was, in the conference program for DrupalCon Boston :

*AMP

Describing the Site Building track at the conference, the program lists track topics, including, "Drupal and *AMP, a systems level view".  The asterisk clearly denotes a substitution variable, which had previously been the constant "L", preceding the abbreviation for Apache-MySQL-PHP/Perl/Python.  This is a clear tip of the hat to Solaris.  The new acronym, presumably pronounced star amp, bestows equal opportunity status to Solaris and Linux within the Drupal community.

I expect there will be plenty of interest in Solaris and other Sun technologies at DrupalCon, which convenes March 3-6 at the Boston Convention & Expo Center - the news of Sun's intent to acquire MySQL did not go unnoticed, and high profile sites running on Solaris are turning up frequently.  I guess that's why my Inbox has been overflowing with correspondence from the Drupal community.

Sun is a Gold sponsor of DrupalCon Boston.  We'll be handing out Solaris Express DVD's and we're giving a Sun Fire T1000 server to the winner of the Drupal Showcase competition.  I'll be a panelist in the Performance Tuning session, and I'll also be presenting some Solaris deployment techniques and best practices similar to what I presented at a DrupalCon last September in Barcelona.  I'm hoping we'll also have a contingent from Project Caroline to let the community in on the next big thing in Web2.0 development platforms.  As if all that from Sun wasn't enough, rumor has it, MySQL Director of Technology, Brian Aker is going to keynote the conference.

The final two days of Drupalcon 2007 were competing with the phenomenal La Merce Festival for conference attendees' time and attention.  Had Sun not paid my airfare to be at the conference I'd have been at the festival catching as many of the 100+ bands playing in Ciutat Vella as I could. 

As such, on Day Three I dutifully presented a session entitled “Industrial Grade Deployment on SAMP” to many of the more prominent developers in the Drupal community.  The session followed an outline that I think helped establish Sun's relevance to the community:

1. How is Sun already connected to the Drupal community?
2. What has Sun been up to all these months/years since you last heard?
3. What are some Drupal specific Use Cases that are interesting on Sun technology?
4. How do you get started using Sun technology?

The slideshow embedded here answers in some depth each of these questions.  Lot's of other presentations from DrupalCon are posted over on Slideshare.net

---

Day Four was capped by some updates and insight from Dries, the father of Drupal and spiritual leader of the conference.  Dries Buytaert created Drupal at age 22.  Seven years later he continues to lead the Drupal community and actively contributes innovations to the platform.  He is a PhD student at University of Ghent where he has published several papers on Java performance, and has developed tools and frameworks for JIT compiler analysis and Java performance tuning.  The irony that Drupal is built on PHP is not lost on the Drupal community or Dries.

One of the most important updates presented by Dries was the 2007 developer survey results.  Sun's value to the community came into sharper focus when he flashed the answers to the question, "What are you good at and what skills do you want to invest in?"  Performance, Scalability, and High Availability were areas in which over 80% had little experience or wanted to learn.

Drupalcon was a great experience.  I'm looking forward to the next one, which is likely to be somewhere in North America this coming spring.

Sometimes the relevance of a technology only becomes apparent after examining it's origins.  For the uninitiated, I'll drop back a few sentences here to help understand Drupal's roots, before talking about some of the interesting things people are doing with it. 

Drupal was created by Dries Buytaert in 2001.  Drupal is licensed under the GPL and is written in  PHP.  It is typically deployed in a LAMP environment although a few prominent sites have deployed Drupal on Solaris.  Drupal 5 is the current release with Drupal 6 due out in fall of 2007.

The popularity of Drupal has been driven by a passionate group of developers who come from very diverse backgrounds and who have applied Drupal to an equally diverse range of needs. 

The early community of Drupal developers grew primarily through rapid adoption by Web 2.0 hobbyists, nonprofits, and political activist organizations.  Howard Dean's push for the U.S. Presidency in 2004, noted for leveraging online tools and communities, owes much of its success to the Drupal powered 'Deanspace'.  This highly visible use of Drupal drew much attention to the community and triggered a huge wave of Drupal adoption. 

One of the most interesting uses of Drupal that I've seen is that which was presented at DrupalCon by Ivan Labra from SPAWAR, the Space and Naval Warfare division of the U.S. DoD.  He is using it as a integration platform in support of SPAWAR's peace and stabilization efforts, in which basic integrated ICT capability must be deployed into austere and sometime unstable environments.  Known as Speed-to-Capability, this project defines a technical architecture and deployment strategy for quickly building communication and collaboration capacity FOSS components combining PBX (Asterisk), Instant Messaging/Chat (Jabber), email (Postfix) and software provisioning (HostMaster) capabilities on the Drupal framework.   In my role advising on technology capacity in the developing world, I hope to work with Ivan in the future to apply this important communication and collaboration capacity.

Day one of DrupalCon Barcelona 2007 is over, but my jet lag is not.  I did manage to stay awake for the entire day, but only had time to attend two sessions:

• OpenID: It's in core... now what?   by James Ransom Walker.   James is clearly an OpenID advocate and says the risks associated with it are manageable, or at least acceptable.  OpenID has been added to the Drupal 4.7, with updates for 5 and 6 coming soon (I'm not sure whether that's and Iraq pullout-style timetable, or a clever call for volunteers to lend a hand - James did say he could use some help).  This much heralded addition to Drupal gives developers an "out of the box" provider and  relying party status if they want it. It also comes with a new set of concerns for developers whose permutations are myriad: What is the trust model I want to deploy?  What level of protection do my users need from my provider service? As a relying party, what level of authentication do I need from a provider?  How do I choose to providers to accept?   Do I care whether a user's ID is globally unique *forever*, or just for now?  The OpenID spec. itself leaves the developer with all of these choices and more.  OpenID's flexibility is both a virtue and a failing.  Maybe someone in the OpenSSO community can lend a hand to James and avert the sedimentation of a partial solution to an omnipresent problem.  OpenSSO is moving quickly to support OpenID provider implementations.  It has support for the relevant federation standards, and it even has a PHP Client SDK and a PHP library for SAML 2.0 Relying Party.  When it comes to Identity Management, I'm not convinced that today's "good enough" won't be tomorrow's compliance regulation headache or M&A due diligence hiccup.  My vote is for an OpenSSO based identity module for Drupal 5 and 6 rather than an OpenID only module.
  

• "Enterprise" Drupal  by Ken Rickart.   Ken works for Morris Communications, a very stodgy, family run corporation.  Drupal adoption at Morris would seem a long shot for the traditionalist culture of this media giant.  But with the aid of Ken's obvious leadership and technical skills, Drupal is shaking things up.  If Ken's experience at Morris is any indication, I'd expect we'll hear about more tremors rippling through vaunted institutions and enduring companies triggered by Drupal's "time to market", low cost advantage.  Ken talked about how he rapidly delivered some high value business services to internal users (contract renewal reports) and external media consumers (online editions of local newspapers) with Drupal, demonstrating just how true the "good enough" axiom can be for certain classes of problems, and why that mattered to the big cheese at Morris whose main functions are to manage the bottom line and shake hands with the pros at Augusta National).
  

I'd better get some sleep.

The People Formerly Known As The Audience landed a piece on Wired.com today about the Open Architecture Network (OAN).  

The story was part of a pro-am journalistic experiment orchestrated through Assignment Zero that produced 80 distinct articles dealing with crowdsourcing, of which 12 earned a spot on Wired's home page.  As one of the OAN story contributors, I bagged my first byline on Wired.com, one in which I was decidedly on the am side of pro-am.  While my contribution was relatively small, it was enough to give me a taste of the post-edit blues - most of my copy was either red-lined, or reduced to the point of inaccuracy.   My original reporting expanded on the technology choices and process of developing a collaboration site using the open source CMS Drupal.   The final piece gives a nod to Drupal and Sun Microsystems, but leaves the wrong impression:

"Even the software powering the site -- designed by Sun Microsystems -- is open source: the Drupal content management system chosen by thousands of nonprofits for its ease of use."

Sun did not design Drupal, and while ease of use is one of the virtues of Drupal it's an oversimplified view of why so many nonprofits use it.  Granted, the focus of the story was not so much on technology as the potential for open source design, but one of the points red-lined from my copy was perhaps one of the most relevant given the context of this crowdsourcing experiment:

"Organizations that use Drupal for their online communities include... Assignment Zero."

Drupal is everywhere.   Drupal's integral role in the explosion of collaboratives born of open sharing and grass roots participation is worthy of an entire Wired issue, if not more than just a mention buried deep in one story.  Even among citizen journalism sites, it's a dominant software platform - witness The Witness Project, for example.

Still, good to see the momentum for Architecture for Humanity continuing in the media - in addition to the Wired piece, another story about the OAN appeared this week in Business Week, and AFH's founder, Cameron Sinclair, was added to the distinguished list of "Thinkers of Tech" for Fortune's iMeme conference next week in San Francisco.

We're still the audience, we just have something to say now. 

 

Planetizen, the online commmunity for urban planners and designers, ranked the Open Architecture Network as one of the Top Ten websites for 2007.  Like the OAN, Planetizen is built on the Drupal CMS. 

While browsing through Planetizen I came across a link to AssignmentZero a crowdsourcing journalistic experiment.  The site is also built on Drupal, and they've got an assignment posted to write about Architecture for Humanity, who they describe as "one of numerous organizations practicing open design".   Along with three other writers, I took the assignment, natch.