Bay Area Startups and Technologies Marina's Web Scale Coral Reef

In case you missed the Oracle BMW sailboat (doubt you did), here it is.

The event itself was quite energizing and technology centric, filled with events, two exhibit floors, and multiple sessions. Exhibit floors included Cloud vendors like Amazon, as Oracle is available on EC2. There was a large presence by Sleseforce.com with their CRM on the cloud and developer cloud Force.com offerings. The Oracle development team had a long stretch of booths on the expo floor as well. Below are a few technology snapshots that particularly apply to Social Networking applications.

To improve performance and scalability of PHP applications, you can pool DB connections with Oracle's Database Resident Connection Pool (DRCP). This is important because by default, HTTP requests are handled by individual process in the following manner. A PHP application is deployed on an Apache server using a mod_php module. Individual HTTP requests are processed within a corresponding Apache child process, which is spawned for each request. All resources, including the database connections, are created and destroyed within the context of that child process. However, creating individual database connections for each HTTP request is a resource expensive operation. Oracle's DRCP allows PHP applications to leverage a connection pool. DRCP keeps track of open connections and recycles them in a fashion similar to a JDBC connection pool. It times out idle connections and a Connection Broker of DRCP manages incoming connection requests. Using persistent db connections improves web application scalability.

Mapping and GIS applications can leverage Oracle's database spatial and mapping technology. With Oracle Locator you can quickly calculate a distance from point A to point B, trace a travel route, and perform amny other mapping functions. Most startups in the GIS space rely on PostgreSQL, but if you happen to use Oracle there are definitely many useful features you can utilize.

Oracle's open source, in-memory database, Berkeley DB transactional data store, is used across a multitude popular Web 2.0 apps. To name a few, MemcacheD uses Berkeley DB as it's core in-memory engine. It's often used as an embedded DB. For example, Sun's large scale, open source, networking platform for gaming and virtual world applications, Project Darkstar, also leverages Berkeley DB. Berkeley DB is also used by Internet Archive's Wayback Machine and Heretrix, an open source Internet crawler project. Ulike TimesTen, Berkeley DB is not designed with a primary goal of SQL query execution, instead it can be used as a key-value pair for large objects. The db stores data in a shallow B+ tree, which contributes to quick data retrieval as well as fast writes. The implementation is available in C and Java.

Oracle's Open Source resources can be found here.

Aside from the Oracle technologies that can be applied for building Social Networking applications, partner applications were a big part of the exhibit. There were multiple startups (in SSE definition) that demonstrated technologies for efficient data transfer, data archival and replication.

Do you have an avatar and what do you think of the 3D world? The FountainBlue had held today it's annual event, that gathered startups, enterprise companies and individuals that are excited about the opportunities in the virtual world application space. The virtual world is getting its adoption in gaming (both desktop and mobile), educational, healthcare, and overall collaborative environment spaces. Players in the space include Comcast, Sony, Sun, and of course startups like Masher Media, ISO Interactive, KohdSpace, among others.

There are multiple technologies involved in building the virtual world applications. Unity is often a preferred choice for 3D game development, Java is often in the list for programming languages, and for deployment Darkstar is used for massively scaled multi-player networking layer. Sun's Project Wonderland demo was focused on how to use open-source toolkit for creating collaborative 3D virtual world, and here is how it looks.

According to Juniper research the mobile app market will be worth $25 billion in 5 years. So which technology would you pick to develop the mobile application. That's the topic Sun and Micello will try to address during the upcoming Sun Startup Essentails Webinar next week.[Read More]

Many of the SSE members are utilizing Sun's Amber Road storage system, while many other startups wonder what it can do for them. The answer is if you have an application that has a heavy use of data such as accessing or operating with the large volume of media files, or an application that requires instant accesses of the persistent data, you may want to consider Amber Road storage systems. Get some insights on the Amber Road functionality and check out the snapshots of the storage management console that is based on DTrace and provides real time analytics of your application storage usage. [Read More]

Most of today's startups are building their solution on the open source LAMP stack or Open Solaris AMP stack. However, some of the startups as they grow and expend, look for commercial alternatives. I've seen startups and smaller companies deploying Oracle back end as their database of choice. Small and mid size companies may be familiar with SAP Business One applications, which is an ERP solution for SMB market. When using the application, often times companies have to customize the application and develop a web based interface for their users to extend the ERP functionality for specified business needs. We've simplified the developer job by easily integrating B1 Web Services into the IDE via the NetBeans Plugin. The Plugin powers up developers with Business One Web Services integration, drag and drop visual editor, and Glassfish auto-deployment features. The details of how to use the plugin, refer to the Building Web Applications with NetBeans IDE and SAP Business One Web Services article published on SDN site.

At this month's LAMP meetup we had an educational talk on Unit Testing by Jakob Heuser, a web engineer from LinkedIn. In PHP, there are a number of frameworks used by developers including SnapTest, PHPUnit, SimpleTest, and Tesrilence. The main principle behind Unit Testing is to validate application requirements.  This includes checking application logic that implements a give requirements, making sure the fault flows are managed accordingly, and various conditional boundaries are satisfied. Unit testing focuses on testing smallest component of the program, such as function, public method, or an include file. With SnapTest, it is pretty straight forward to get started. After implementing PHP classes to test the application you can start multiple tests simultaneously to expedite the testing process. While SnapTest has a low starting barrier, testing frameworks like PHPUnit is a much more comprehensive framework. But regardless of the framework that you pick, here is an ideal order in which you should be building and testing your application:

1. design your class/API
2. create a test suite
3. implement the class/API
4. run the test suite
5. fix failures or errors and go to #4 again

After all test suites were ran and errors have been fixed, it's a good time to do refactoring and simplify the application logic, create a more elegant implementation, and then obviously run the tests again to make sure that newly introduced logic doesn't break anything else in the program.

Jakob maintains and contributes towards SnapTest framework and his presentation is available at:
http://www.slideshare.net/Jakobo/unit-testing-first-steps

Expanding iPhone developer community had their monthly meetup in June in Palo Alto. A couple iPhone apps, OfferPal and TapJoy, were followed by the book announcement on Cocoa Design Pattern.
[Read More]

The last two meetups of the Greater SF LAMP community that we host at Sun campus were focused around PHP development. The December one had an exciting topic on Object Oriented Design Patterns for PHP presented by Robert Gonzalez of Bay Alarm. What's interesting since I've started working with the Web 2.0 community I rarely heard the word pattern that we traditionally apply in the sw architecture, the main emphasis is on functionality and the business ideas fostering the implementation. However, with PHP 5 you can apply the Object Oriented principles and achieve an elegant application architecture, well beyond 'just getting the job done'. The speaker went over a few patterns that I'm used in the Java world, and shown examples of how to apply them in PHP. The traditional Singleton and Factory patterns were discussed. Also the Strategy Pattern was an interesting one as it encapsulates a set of algorithms that change and leaves out what doesn't change. This pattern makes encapsulated algorithms that are interchangeable. It favors composition over inheritance. A ComputerPlatform interface can have multiple implementations such as ComputerPlatform_Mac, ComputerPlatform_Solaris, ComputerPlatform_Windows that all implement a given method, but individual implementation may vary. All in all it was a great session and there are already quite a few books out there that discuss patterns with PHP as the language matures.

This month the two speakers covered NetBeans IDE and Aptana's IDE around their PHP support. So from design to implementation, if you attended both of these sessions, you're armed with all you need to create PHP applications.

Bebo's Developer Challenge is open for another few weeks, so developers can submit their applications till January 6th. While Web 2.0 developers with PHP and Ruby already looking into the contest, this is a great opportunity for Java developers to move towards Social Networking realm and submit their applications based on Bebo's Java APIs. What I like about Bebo development is that they have a pretty neat API Browser that allows you to validate the  API call before incorporating it into your program.

With Sun Startup Essentials, you get a year free Joyent accelerator for the Bebo application hosting.

There are also a couple exciting technology announcement from Sun around JavaFX and OpenSolaris 2008.11 release that generate a good bit of momentum in the community. Check out the summary of the latest features in the OS and as for JavaFX you happened to be on Facebook, don't forget to play the puzzle, it's pretty cool.

From Web 2.0 to Holiday 2.0. Ready for a holiday bash? Sun Startup Essentials is co-sponsoring Holiday Party 2.0 - Show Me the Money! Don't miss the event which is coming up this Friday in San Francisco, see details  and RSVP if you're planning to attend. See you all there.

This year's Code Camp attracted somewhere between 700-900 developers, much bigger compared to the last year one. I had a chance to present on both days of the event. The Sun, Yahoo, and Bebo panel focused around discussion on Scaling Platform with Bebo, Sun, and Yahoo on Saturday Nov 8 that has been recorded and posted. The idea behind this panel is to outline some of the key criteria for building scalable and reliable Web 2.0 applications. Steve Cohen, a head of engineering for the Bebo platform, outlined MVC based Java framework enabling the Bebo site. It was pretty exciting to see Java as a key technology powering up one of the top Social Networking site, that has around 40Mln users across the globe. As you can see in the podcast, Steve outlined their core architecture, benefits, and challenges that they're facing with increasing the scalability requirements. On the Sun side, I had a chance to to present some of the key aspects of building, deploying, and scaling a Web 2.0 application. Sun's tools and technologies including NetBeans, Glassfish, AMP Stack on OpenSolaris, MySQL, xVM, OpenSolaris, DTrace, and ZFS are essential componets of the Web 2.0 application development and deployment environment. Tom Huphes-Croucher, a tech evangelist from Yahoo, outlining open developer APIs and providing some insights on Yahoo scalability story. We had great questions from the audience on how to optimally manage, test, and prepare for the large scale.

Silicon Valley Cloud Computing gurus, SOA experts, and Virtualization vendors had a chance to explore and share the technology insights during the Cloud Computing Expo event in November. I attended the Cloud Computing Bootcamp, led by Alan Williamson, as well as preview products and offerings at the expo floor. The bootcamp was quite informative and started with the definition of Cloud Computing term. Well, there is no formal definition commonly shared in industry, unlike Web 2.0, and it is very broadly defined as on-demand provisioning of application, resources, and services, that allow resources to be scaled up and down.  Wikipedia's definition, authored by Sam Johnston, specifies a six layer stack that encompasses Client, Services, Application, Platform, Storage, and Infrastructure. That was the basis for the bootcamp session, with individual layers explored in depth with regards to their nature, pros & cons, and players in each space. I'd like to summarize just a few of the layers and mention Sun technology in each category.

Layer #1 is the Infrastructure Layer. It consists of an operating system on demand, i.e. cpu/memory/processing. Some of the vendors in this space include Amazon EC2, FlexiScale, GoGrid, AppNexus, and Joyent. Sun partners with a number of hosting vendors to enable OpenSolaris as part of the deployment platform, which is available with Amazon EC2 and Joyent among others. With OpenSolaris, you can leverage DTrace, optimized AMP stack, and ZFS. Additional value add for Sun Startup Essentials members is FREE one year hosting available for Facebook, Bebo, and OpenSocial application developers on Joyent.

If you are a hosting provider, Sun offers all the building block for setting up the hosting environment. Starting with servers, storage, virtualization, OS, management components, AMP stack, you can leverage many of the proven Sun technologies to build up a cloud environment.

On the Infrastructure layer management side, RightScale and ElasticServer are the top two players that help you to monitor your site, provision new instances, and manage your environment. Similar to other layers and vendors in the space, there is no out of the box interoperability among the infrastructure providers. Therefore, your application stack should be very portable so that if you decide to move from one hosting environment to another, you can do so.

Layer #2 is the Storage Layer, which includes file or object based data storage. Users are billed based on the bandwidth and consumed storage. Amazon S3, Nirvanix, Mosso, Amazon's SimpleDB, Google's BigTable, Azure Storage are among storage providers in this space. With growing number of Web 2.0 companies offering media based applications, e.g. content aggregation, image searching and processing, these services are quite popular. On the down side, there are no common storage data access API shared among the vendors; also, there are limitation on the size of the data. As your storage consumption growth, the bandwidth cost of the service provider may become higher than the cost of the custom managed storage infrastructure. Thus, it's important to monitor and provision storage in a way that is cost effective for your business.

Sun has introduced recently Sun Storage 7000 series, and with Sun Storage 7210 Storage Array (SS7210) in either striped or mirrored configuration you can stream over 700MB/sec video content with 90% CPU utilization.

On the storage management side, Jungle Disk  (acquired by Rackspace), ElephantDisk, PutPlace.com are a few popular service providers.

Layer # 3 Platform Layer comprises a complete software stack and an IDE for the cloud. This layer is responsible for the load balancing and resource provisioning. A few players include Google App Engine with support for Python, Force.com (SalesForce), MS Azure for .NET developers, Heroku.com for Ruby On Rails gurus. The latter one is built on Amazon's EC2.

In this layer, Sun offers to developers Zembly platform that enables building and deploying Facebook, OpenSocial, Meebo and iPhone applications.

With great benefits offered by each of the providers, there are still deployment differences, vendor lock in, and limitations to the underlying language support.

Cloud Computing paradigm became a de facto deployment environment for many startups, but before jumping on on its bandwagon, it is important to keep in mind your application requirements from the infrastructure, platform, database, performance, and scalability standpoint. The greatest value of the Cloud Computing is reduced TCO for application deployment and management, thus many companies take advantage of a cloud particularly for the initial launch of their sites. Once user based grows, some companies realize that for their specific application and data usage, cloud many no longer be the most effective alternative from the cost perspective. Aside from that, Cloud Computing may not offer the same server configuration, security, and SLA's critical to the business, as you may setup in your datacenter. Cloud provides you with a sandbox for your application deployment, thus flexibility is limited.

Bebo hosted a developer event last night with presenters covering topics on building Bebo apps, monetizing the apps, porting Facebook apps to Bebo. The good news for FB developers is that Bebo platform is the F8 compatible, so if you have FB apps you can easily port them to Bebo and with Bebo user demographics primarily being in Europe you can expand your userbase across to the European community. Blake Commagere, the creator of Zombies, Vampires, and Werewolves games on Facebook gave an interesting talk on migrating to Bebo platform. It took him only 9 hours to migrate 4 FB apps to Bebo.

During the event Sun announced free Bebo application hosting with Joyent, so if you're building Bebo app, don't forget to take advantage of this one year free hosting offer. Bebo announced a developer context for the most engaging app, check it out.

 Here is a complete recap of the event posted by Sudha Jamthe

Bebo DevNite is coming up at AOL campus with exciting talks, demos, and social extravaganza accompanied by pizza and Wii. I'll be presenting on how to develop a reliable and scalable Bebo application using Sun's open source technology. Just in case you're new to the Bebo world, Bebo is a social networking site that has been quite popular in Europe and now gaining momentum in the US. Now you know what Bebo is all about, you can take a look at Bebo APIs, which are supported in Java, PHP, and Ruby, so you can use to develop a Bebo application. Using those APIs and Sun's technologies including MySQL, OpenSolaris, Glassfish among others you can build a social networking  application that appeals to your Bebo friends and the web audience in general.

There is an interesting article on Olio toolkit on High Scalability site that brings up a very good point about evaluating different technologies before deciding which one is most suitable for your application needs. There is a common trend that I see among new gen startups, where PHP developers starting their new companies and using Ruby on Rails over PHP due to its performance, ease of use, and rich functionality set. With every application there are different factors that need to be accounted for. How do you select technology, database, underlying deployment environment, and how do you know that it is indeed the stack you can reliably deploy your application on? Olio allows you to validate these questions. Interestingly Apache's Olio evolved from Sun's Web2.0kit, and now there are three implementation of it in Java, Ruby, and PHP. When it comes to optimized Web2.0 stack, the OpenSolaris community use the Webstack which embraces multitude of components such as Apache, MySQL, PHP, Drupal, and allows developers to leverage performant components of a traditionla Web 2.0 applications on OpenSolaris. You can learn more about Webstack and become a community contributor.