Alfred C. Chen's Weblog

Recently, we released a new feature to the download platform that powers the Sun Download Center.  Internally, we refer to it as the Advanced Download Widget (ADW).  Essentially, it's a Web component that can be deployed on any Sun-branded Website, to deliver an integrated and streamlined download experience.  Working with Lifecycle Marketing, we have also integrated the ability to present the user with a free offer (e.g. whitepapers, training, etc.) that complements the download.  It's completely optional, but to receive the free offer, the user will need to login with a Sun Online Account or create a new one.  To see it in action, checkout the Java ME SDK 3.0 download page.  Although the project was generally successful, there were some lessons learned (in terms of went well and what can be improved from my own personal perspective) that I would like to share via this blog for future reference.

When developing new Web functionality that relies heavily on Web browser technology, it's important to understand your users.  Do they largely run on a single Web browser / platform combination (Intranet apps), or do they span the gamut in browsers and platforms used?  The answer may greatly affect your project plan and testing strategy, so find out before you start on your project.  Sites such as Wikipedia publish aggregated Web browser usage stats for the Internet, but it is better to take your own measurements if possible.  Our sampling of a very popular Java download yielded slightly different distribution with 48.6% running Internet Explorer and 40.8% running Firefox.  The data helped shape our testing strategy.

Given the constant evolution of the Web browsers, it's not always practical to maintain backward compatibility to outdated Web browsers; however, forward compatibility for new releases should definitely be a priority.  Internally, you want to establish guidelines on the Web browser makes and versions, as well as the browser platforms that you can support.  This way, you can provide Engineering and QA with clear expectations on the testing scope and staffing needs.   It's also good to have designated personnel who keep tracks of the product roadmap for key Web browsers.  During the development phase for the ADW, new version of Firefox (3.5) and Safari (4.0) were released, but they were not on our radar.  We later uncovered some minor incompatibilities with these Web browsers during the testing phase that prompted additional round of testing and contributed to some avoidable schedule delays.

One key aspect of project management is accurate planning of the time and resource it takes to complete the project.  While the Development and QA phase are largely determined by Engineering's estimates, the business generally drives User Acceptance Testing (UAT) and therefore estimate the resources and duration required for UAT.  Because the project scope varies from release to release, successful UAT planning requires a good blend of gut feel (art) and common sense (science).  By applying past experience and intimate knowledge of the system to the bug fixes and enhancements in scope, you can devise a rough approach to the UAT test plan and test cases.  From the estimated time for completing each test case and the availability of UAT testers, you can then derive the duration required to complete 1 round of user testing.  When testing new features that relies heavily on Web browser technology, be sure to add extra time and/or testers for targeted cross browser/platform testing.  Finally, allow time for bug fixes and at least a second round of user testing to make UAT a success.

Finally, while the project life cycle ends when the release goes live, the product life cycle continues on.  By product, I'm referring to the system or the download platform in this case.  Although it's possible to roll out new features that meet the user's needs on day 1, quite often the road to nirvana involve a couple design iterations.  To avoid the design-in-a-vacuum pitfalls, it's essential that you have ways to collect feedback and gain insights into the user's real world interactions with your system.  In our case, we chose to conduct a usability study to gather feedback from a diverse group of users whose experience with Sun ranges from none to developers and Sun customers who are quite familiar with our Websites.  Although the usability study was insightful, we plan to integrate Omniture into future releases so we can measure and assess the usability of the ADW across the entire user base.  Meanwhile, we have other ways (see my blog on The Voice of the Customer) to collect and act on customer feedback as well.

Do you use Google to search the Web?  Isn't it great that Google returns a search result of relevant information for just about anything that you enter.  In software engineering, there's a term that describes such predictable and reliable outcome.  It's call the "happy path" and it defines the default user experience when the software works as intended.  I first learned about it in college.  It has been a while, but I think the subject still has a lot of relevance today, especially in Web application design.

Imagine your reaction if Google errors out or returns no search results, every other time you use it.  That would not be a good user experience.  So how do you factor in the happy path into the design and the implementation plan for your application?  Whether you are building a Social Media Website or an IT application for internal consumption, the process starts with good product or business requirements.  When you write the high level requirements and use cases, you are defining the happy path.  Good requirements should specify how the application process the end user's request to deliver a user experience that meets or exceeds the end user's expectations; focuses on the desired functionality.  In the case of Google Search, I'm guessing that the original requirements may have been written as follows:

Build a search engine with a simple user interface that accepts the user's input, intelligently determine the most relevant Web resources, and present the user with the search results sorted by popularity.

Good requirements also facilitate great design.  When evaluating your design options, you should focus on a design that engages the user and keeps the user on the happy path.  Some Web applications calls for rich interactions (RIA).  In the case of Google Search, as simple design seems to work very well too.  Either way, a streamlined user experience will definitely appeal to the users and enhance conversion rates.  Your design should also account for how the application recover when something goes wrong.  There's a popular saying "Sh*t happens", and it definitely applies to anything on the Web.  Most users will tolerate some mishaps,  but you really should do your best to minimize the impact to the user experience.  Your average Joes won't put up with HTTP 404 or 503 errors, but they will appreciate good humor such as the case with the Twitter Fail Whale.

Finally, how do you ensure that you are on target with your design?  It is important to test out the new functionality and verify the outcome against the original requirements.  In-house User Acceptance Testing (UAT) is a common practice for IT applications, and external Web applications without established communities.  Since UAT involves a controlled group of testers (generally superusers), be sure to provide all users with a feedback loop.  Public beta testing and user-driven designs seem to be very popular among Web 2.0 applications.  GMail which was launched in 2004 has retained its beta status despite its success and mass adoption.  Facebook with over 200 Million users engages the user community through the Facebook Blog.  Both are tremendously successful in attracting and retaining users.  Although there's not a singular approach to engineering great solutions, there's definitely one common theme:  Stay on the happy path.  After all, who doesn't appreciate software that simply works.

One way to develop a better understanding of the customer's needs is to simply ask for their feedback.  Through out sun.com, you will notice a floating [+] icon in the lower right corner of your Web browser window.  When you click on this icon, an OpinionLab scorecard pops up allowing to you provide input on your Web experience as it relates to the content, design and usability of the current page.  Below is a screen snapshot of a sample scorecoard:

Recently, we added the OpinionLab icon to the Web pages for the download application that powers the Sun Download Center.  In the month of April alone, we received 481 ratings through the OpinionLab scorecard.  90 people chose to provide additional feedback through the Comments field. The overall rating was 3.5 out of 5.  It was a bit lower than I expected.  Again, this is why we're asking for your input, so you as the customer can help drive further improvements base on your needs.

I would like to thank you if took the time to provide us with your feedback on the Sun Download Center.  While I cannot share the specifics of the feedback that we received through the Comments field, I can say that most of the comments were fair.  I'm still flattered by the number of people who are fans of Sun, our products or the Sun Download Center.  Regarding the constructive feedback related to the download experience, if it's actionable, we will definitely include the enhancement in one of our upcoming releases.