Wednesday Sep 30, 2009

Yesterday, we deployed build #82 of the Sun Software Library into production, and two weeks ago, we deployed build #81 into production, enhancing the library functionality in the following areas:

  • New Artwork: We have some new artwork:
    Sun Software Library artwork
    Other sites that want to link to the Sun Software Library can use this artwork.

  • Request to be a Content Manager includes a message: One of the reasons we had to change our Terms & Conditions is to allow the users to better communicate with other users.  In this release, we have added the ability for a content management requests to include a message, enabling the user to tell the current content administrators why they are requesting access. ⁞ For small organizations, "where everybody knows your name", this may not seem like a useful feature, but for large organizations (e.g. think of companies the sun of Sun Microsystems), this is especially useful.

    Sun Software Library Content Management Request Message

    The next few releases will include more functionality, enabling users to communicate with one another via the library.

  • New Terms & Conditions: As mentioned earlier, we wanted to add new functionality in the Sun Software Library, enabling users to anonymously contact publishers (and vice versa).  If you log into the Sun Software Library, and have previously accepted the terms and conditions, you'll see this popup, informing you that new terms and conditions are in place:

    Sun Software Library Terms and Conditions have changed

    Click "Accept" to continue.

    We've added this language to the Privacy section (section 6).  You can view the entire terms and conditions at https://library.network.com/terms.html.
     “You acknowledge and agree that Site Offerings may enable You to
    allow to be contacted by Sun other Users, and may enable You to
    submit personally identifiable and other information to other
    Users.   You consent to such contact by other Users (unless Sun
    expressly gives You notice and an ability to opt-out from such
    contact), and to the unrestricted, non-exclusive use by such
    other Users of any such information You choose to submit to them,
    and You assume (and Sun disclaims) all liability and
    responsibility arising therefrom or related thereto.”
  • Internet Explorer support: IE is our most popular browser, some 55% of our users use it.  As such, we need to support it - all 3 versions of it (IE6, IE7, & IE8).  However, we do pay for that support, in terms of engineering time, both in testing, and in fixing bugs.  Primarily because of Firebug and our Mac desktops, Firefox continues to be our primary development and debug platform, with Safari being a close second.  We do fully test on Windows platforms, but find that what typically works on the other two browsers breaks on IE.  Recently, we've spent a significant amount of time debugging IE8, while maintaining functionality for previous versions of IE.

    It's hard to estimate what our actual cost is for supporting IE6, but it is relatively high.  As such, we support efforts to get users to migrate away from IE6.

  • We are 100% JSLint "The Good Parts" compliant: After we integrated JSLint with NetBeans, we got to work making sure our own Web UI (written in JavaScript) passed JSLint.  This took us a while, given the amount of JavaScript code that we have.  The first decision we had to make was what options we wanted to JSLint to enforce - we decided to leverage The Good Parts, written by Douglas Crockford.  Given that this is our entry in build.xml:
            <jslint4java haltOnFailure="${cqs.build.jslint.failonerror}" options="${cqs.build.jslint.options}">
              <formatter type="plain" />
              <filelist id="jslint.filelist" dir="${basedir}/web" files="${cqs.js.jslint.files}"/>
            </jslint4java>
    
    We then set this option in our project.properties file:
            cqs.build.jslint.options=white, onevar, browser, undef, nomen, eqeqeq, plusplus, bitwise, regexp, strict, newcap, immed
    
    After we cleansed our JavaScript code, we set our builds to fail if JSLint detects any errors:
            cqs.build.jslint.failonerror=true
    
    This approach has already paid a dividend: One of our developers wrote some code that worked perfectly on Firefox and Safari, but would have created issues on IE, due to IE's more stringent syntax checking.  Normally, we would have caught this error during our Quality cycles, but JSLint picked up the issue by causing our Continuous Integration build to break.  The issue was fixed in minutes at a relatively low cost, rather than in hours at a much greater cost at a later period in time.

    This incident alone has validated our investment in integrating code inspection tools into our Continuous Integration builds.

    And, no, our feelings weren't hurt!  Humbled, maybe, deflated our egos a bit, definitely, but not hurt.

  • Cobertura integration: In the spirit of integration of code inspection tools into our environment, we integrated Cobertura into our Continuous Integration system.  We've run Cobertura before, but always manually, on an as needed basis.  Now we always have an up to date code coverage report.  Combined with Hudson's Cobertura Plugin, we configured our builds to measure code coverage taking into account all of our automated testing: Web Services based functional tests, Selenium based Web UI tests, and jUnit based traditional unit tests.  This way, we can measure the effectiveness of our testing strategy in realtime.  We still have some work to do in this area, but at least we know where we stand:

    Sun Software Library Code Coverage

  • And as always, lots of bug fixes and minor enhancements here and there...
There are more improvements in store, keep the feedback coming!

Tuesday Aug 25, 2009

Yesterday, we deployed build #80 of the Sun Software Library into production, enhancing the library functionality in the following areas:

  • Additional search options: you can now limit or filter search results based on a tag.  For example, you can now search for all entries that have a particular keyword and are tagged with a particular tag.  To do this, when you open up the "more options" dropdown right under the main search area, you'll notice a new input field titled "Tag".  In this field, you can type in a tag name that you want to use to filter the search results.  This field is "auto-completing", if you type in the first few letters of a tag name, you can select the full tag name from the dropdown list.

    You can see this additional functionality in action:

    Sun Software Library Combined Keyword and Tag Search capbility

  • Improved context sensitive help: We have also improved our context sensitive help functionality in the add/edit popup.  Specifically, we've automated the functionality of copying the FAQ questions from our Sun Software Library help center to this panel, improving the overall quality of the context sensitive help available.  This is another example of how we strive to automate as many of the manual processes that we have, so that we can provide a better service at a reduced delivery cost.

    This image displays the help you get when you edit a publisher entry.

    Sun Software Library Context Sensitive Help

    This image shows the help that you get when you select the "tags" tab.

    Sun Software Library Context Sensitive Help

  • Bug fixes - as always, we fixed bugs.

There are more improvements in store, keep the feedback coming!


Monday Aug 10, 2009

Earlier today, we deployed build #79 of the Sun Software Library into production, enhancing the library functionality in the following areas:

  • Most viewed date range: You can now select which time period should be displayed in the "most viewed" area of our home page, so you can see what's popular over different date ranges. 

    library most viewed date selector

    The default setting is "all time", so you'll see what's popular since the beginning of the Sun Software Library, since this is the view that was presented all along.  If you want to see a different date range, simply change the setting.

    There are two aspects of this date range that are worth noting:
    • The data for this is updated once a day.  Our "most viewed" data does not change all that often, and we didn't think it warranted the extra processing required to make this more real time.  If you disagree, tell us.
    • We keep the data for month boundaries only, so when you select "Current Month", you'll be seeing the "most viewed" entries for the current calendar month, not the past 30 days.  For example, if you select "Current Month" on August 10th, you'll see the most popular entries for the month of August (e.g. the last 10 days only).
  • Upgraded our Web UI to ExtJS 3.0: As mentioned elsewhere in this blog, we use the Ext JS Cross-Browser Rich Internet Application Framework widget set for developing our Web UI.  Ext JS recently released version 3.0 of their library.  We upgraded our Web UI to this new version, found some bugs along the way.  The new library enabled us to support more browser types.
  • Support for IE8 & Chrome: We now officially test and support Internet Explorer 8 and Google Chrome browsers for our web site.  IE8 has some significant improvements for JavaScript developers, to quote Joe Hewitt (creator of Firebug): "I couldn't be happier that Microsoft completely copied Firebug for IE8."  We agree, debugging JavaScript on IE8 is finally reasonable.
  • JSLint: We have incorporated JSLint, the JavaScript code quality tool written by Douglas Crockford, into our NetBeans and Hudson based development process, as described in Ari Shamash's blog.  I typically agree w/Douglas Crockford, but JSLint did not hurt our feelingsWe are human after all, we welcome tools that make our lives better and our development more efficient, as I've mentioned several times in this blog....
  • Lots of bug fixes, as always.
There are more improvements in store, keep the feedback coming!

Thursday Jul 30, 2009

We recently deployed build #78 to the Sun Software Libray, the focus of this release was to improve the experience for publishers in a couple of different areas.

Here is what we added:

  • The primary feature is Context Sensitive Help while managing content.  This feature is easier to describe via a short screencast (less than 2 min), so please press the play button, below.
  • We also fixed numerous bugs associated with logos on the web site, for example, logos of multiple different dimensions are now handled properly.
Usage for the library continues to increase dramatically, we want to thank you for all of your support and kind words.  Please keep the feedback coming.

Monday Jun 29, 2009

Earlier today, we deployed build #76 to the Sun Software Library.  For those of you keep up with our numbering scheme, you'll notice a jump in our build numbers.  We were occupied with a side project, one that hopefully we'll be able to show at some point, but since JavaOne, we've been back working on improving the Sun Software Library full time.

Here is what we added:

  • Browser Based WYSIWYG editor for descriptions & reviews: we've finally joined the 21st century in implementing this functionality.  To prevent various cross-site scripting security issues, we only support a subset of HTML for the description field:

    Browser based WYSIWYG Editor

    Formatted reviews can easily be created:

    Browser based WYSIWYG Editor

    For those of you that want to still edit the HTML, you can easily switch back to that mode using the rightmost button in the editor toolbar.  As an aside, we also fixed a bug in the logo preview section of the "Basic Information" tab.  Previously, under certain circumstances, the log was being distorted.

  • Tag Management improvements:  Tagging entries with the right tags in the Sun Software Library will make the entries easier to find and use.  Based on user feedback, we've revisited the "Add Tags" section of our Web UI and improved it as follows:
    • You can now add a "most used' tag directly to your page:



    • The "other tag" field is now easier to navigate, as the search results all show up in one list:



  • Most Recently Updated section of the home page is now more accurate.  Previously, an entry was considered "most recently updated" only if the entry itself was updated (e.g. the description field was updated, or the name was updated, etc.)  Now, an entry is considered "updated" if it or any of its associated data is updated.  For example, if somebody adds a software version to a software record, or if somebody writes a review, that entry will be considered "updated".  Kudos to the "Tech Tracker" team for making this request.



  • Help Center UI improvements: Our help center finally got a much needed face lift, including styling to make it look like our primary site, as well as an embedded screencast orienting new users to our app.



We've also fixed numerous bugs, etc.  There are more improvements in store, keep the feedback coming!

Thursday Jun 18, 2009

I thought I would anonymous share some of the feedback we've received recently regarding the Sun Software Library:

  • "Very good looking user interface, I mean the Sun Software Library webpage, congrats." and "Site is well laid out and easy to use." - these are the ones I'm most proud of, especially given all the flack we've received along the way regarding our Web UI.  This is somewhat ironic, since we initially designed the site to be purely a set of web services that others would use to build UIs - we never planned on building a real UI.  Our initial UI's were really just "reference implementations", which many mistook to be production quality Web UI's.  Funny how things evolve.  You can read a bit about our original ideas here.
  • "It would be cool if the user can access to an application live demo, obviously web based apps." - the publisher can always associate a screencast with the software demonstrating how the software works, this screencast would show up at the bottom of the software detail page.  But even better, we're working on a feature that would allow the publisher to upload a Virtual Machine Image (e.g. VMI in OVF format) that users can instantiate and use (either on their desktop, e.g. via Virtual Box, or on a public cloud).
  • "i am regular visitor of this site" - awesome, we like to hear that!
  • "in the morning i regularly checks this site." - so do I, but only after my coffee fixes (most days, one cup just doesn't do it).
  • "This site keeps me Updated."
  • "i can get all the news and technical stuff from here."
  • "I want to wirte a review to get a free T-shirt, but I found the web  did not work!I try serveral times.The System always  displays  a error mesg  saying "a inner error please try again the later" when i set my screen name. you know I can not write review unless i set my screen name." - looks like there is a bug in the section of our WebUI that enables setting the screen name.  We are working on a fix.  Until then, you can use this page to set your screen name: https://reg.sun.com/createscreenname.

Keep the feedback coming, folks, we always want to hear from people who are using the Sun Software Library.

Monday Jun 01, 2009

Write reviews for your favorite applications in the Sun Software Library and get a free t-shirt*.

For details on this promotion see:  http://developers.sun.com/community/softwarelibrarycontest/

 *** Please note that writing a commnet to his blog entry does NOT qualify you for a free t-shirt. Please read the instructions provided in the link above carefully.

Wednesday May 27, 2009

Bob May on the Sun Software Library team put together this overview screencast, demonstrating how to search and publish using the Sun Software Libray.  It is very nice, and in less than 10 minutes, shows all the prominent features.

Of course, this screencast is also visible directly inside the Sun Software Library: https://library.network.com/app.jsp#resource/5236

Friday May 01, 2009

Our latest version - build #71 - was deployed into production yesterday.  Here is what we added:

  • What's New: This new panel on the home page takes a subscription feed from our blog at http://blogs.sun.com/thelibrary  and displays the three most recent entries right on our home page.
    What's New and Featured
  • Featured: This new panel on the home page allows us to highlight specific publishers, software, or resources directly. Want to be listed here? Give us feedback.

  • Searching for a specific tag: We have a lot of tags in our system - approximately 3000 of them.  The vast majority do not make it into our tag clouds.  We now allow you to search for a specific tag, or set of tags, using the same mechanisms as keyword search.  This way, if you create a tag that is interesting to you, but doesn't make it into the other tag clouds, you can find it.  The output is a little different, however - we display tag search results in a custom tag cloud.

    Searching for Tags

    Please note that this feature is different than using a Tag to search for entries in the library, which has been in the system since the very beginning.

  • Logo Preview in add/edit panel to improve usability of logos: Previously, many entries in the library did not take advantage of our logo mechanism, we always wondered why, since after all, an entry with a logo looks better than an entry without a logo.  We always wondered why our users do not take advantage of this feature.  Other users tried to take advantage of this feature, but put in invalid URLs.

    Many principles in User Interface design teach that if the user does something wrong, the sooner you inform the user, the more usable your application becomes.  As such, we now display the logo preview directly in the panel that allows you to edit that particular field.  The response is in real time - if you type in an invalid URL for your logo, this is what you'll immediately see:

    Invalid Logo URL

    Type in a valid URL, however, and you'll get some instant gratification!  Ah, the sweet site of success ;-)

    Valid Logo URL

    The circled annotations above were created manually using an image editing tool - the app does not try to figure out what is wrong with the URL, that's a pretty hard problem to solve, we'll save that one for the next iteration. ;-)

  • Improved content management requests and authorization emails: The emails that we used to send whenever a user requested to be a content manager, or when that request was approved or denied, needed some improvement.  We paid some attention to them, and now they are significantly more user friendly.  Our application has direct URL's to many portions of the app, we take advantage of this by including a URL in the approval emails directly to the approval section of the application.  This is typical in traditional web applications, which have a URL for each page.  Our app is a RIA - which exists on a single HTML page, making this issue a bit more complex from an engineerign perspective.

  • Prettier URL: While we're not that vane, our old system used to include "CatalogQueryServer" as part of the URL structure.  CatalogQueryServer is the web context at which our application is deployed.  This used to leak through sometimes in our URL's, making them less user-friendly.  Specifically, our system used to redirect users from http://library.network.com  to https://library.network.com/CatalogQueryServer  (note the HTTP to HTTPS transition in the redirect), and the web context used to leak through the automated redirect.  We have fixed that.  So now we have a nice pretty compact URL structure.

As always, feedback (either positive, constructive, or what the weather is like where you) is always welcomed!

Thursday Apr 30, 2009

Try out the new tag search feature! Search for a tag with keywords by typing the keywords in the search area and selecting "Tags" from the "Scope" drop down list. The results will be shown as a tag cloud in the "Tag Search Results" tab. Click on any of the resulting tags to find items tagged with it.

With so many tags being added daily to the Sun Software Library, this is a great way to check out the specific ones you are interested in. Especially when they do not show up in the most used tag cloud or in the other available tag clouds: Industries, Locations, Solution areas and Sun Technologies. 

If you are adding a tag it might be good to first search for it, or variations of it, to see how it is being used in the library. Please keep in mind that tags with multiple words and spaces are allowed in the Sun Software Library.

We look forward to your feedback about this new feature.


If you find any inappropriate or incorrect content please let us know by sending an email to ndc_library_content_challenges@sun.com with the following information:

Subject:  Inappropriate/incorrect content

  • Your name
  • Location of the content, including the URL
  • Your correction

After receiving the email and examining the content, appropriate action will be taken within 2 business days.

We appreciate your help in keeping the Sun Software Library content current.

Tuesday Apr 21, 2009

On Monday April 13th, 2009, we deployed build #69 of the Sun Software Library.  This blog entry details the additions we've made.  If you've been following this blog, you know that the Sun Software Library engineering team does a great job of deploying a new build every two weeks, but the blog entries typically trail the deployment a bit.  That's completely because the writer of this blog is a slacker.  No other excuse can be made.  All sorts of gimmicks were tried, apparently to no avail.

This release completes, for now, all of our startup performance improvements associated with our RIA.  As you know, the Sun Software Library user interface is written in JavaScript, and accesses the data in the library using Restful Web Services.  Our Web UI currently has:

  • 12,000 lines of JavaScript code written by us, in 17 different files
  • 2,000 lines of CSS code (I guess you can call CSS code, right?), in 4 different files
  • Oodles of images
  • Many 3rd party libraries (for example, we rely heavily on the Ext JS toolkit for web UI components)

When we initially deployed the application many many months ago, the browser had to download each and every one of these components separately, requiring approximately 20 seconds to load the entire application, and that was on a fast Internet link in relative network proximity to the web site.  Performance from distant parts of the world was atrocious, and that's putting it mildly.  We made things a bit better by building an "index.jsp" home page that loaded relatively quickly (e.g. approximately 8-10 seconds), then transitioned to the "app.jsp" main page (which took approximately 20 more seconds to load).

So we looked for tools to help us improve performance.  After all, we're not the only ones in the world writing RIA's in JavaScript.  But amazingly, we didn't find any tools that fit our way of doing work:  we're LAZY, and we're HUMAN!  Being lazy means we abhor labor intensive, manual processes since they are typically very error prone, whereas being human means we like using tools.  Since we couldn't find the right tools, we went about integrating/building various components.  We'll include details in a future blog entry.

So where are we today?

  • A single "app.jsp" page that contains our entire application.
  • A single CSS invocation,  which is automatically generated with a unique file name containing a timestamp, so it can be cached by the browser but also updated in real time.
  • Two JavaScript invocations, one for Ext JS and one for our entire application, each of which have unique file names similar to above.
  • Most of our images are integrated in to a Sprite, automatically generated.  The only remaining images that are distinct come from the 3rd party libraries we are using.
  • All of our content has a "far future cache expiration date".  A browser with a primed cache only requires 8.3 KBytes of new content to render the application (that's the web services call to retrieve the "most viewed" and "recently updated" entries).

Net net, our entire application loads in approximately 2.5 seconds.  And since it is JavaScript based - once the application has started - only web services calls are made to retrieve data to display.  While there is still some work to do to improve performance even more, we're pretty satisfied with where we are.  The only caveat to the home page load time is that the logos for the various entries are not (yet) cached by our service, we retrieve logos from the 3rd party sites on each invocation.  This makes our page load time dependent on services other than our own.  We measured the page load time w/remote logo loading turned off, using yslow.

 So what else is new? 

Sun Software Library Home Page

Our home page now links to three key Sun organizations that we work with daily:

  • Sun Developer Network: Becoming an Sun Developer Network (SDN) member makes you part of a vibrant worldwide community of developers, and gives you access to cool stuff and exclusive offers.
  • Sun Partner Advantage program: The Sun Partner Advantage Program connects Independent Software Vendors (ISVs) to free or deeply discounted offerings as well as sales relationships, so they can deliver solutions/services to an expanded market.
  • Project Kenai: Sun's new foundation for its connected developer experience of tomorrow.  Freely host your open source projects and code as well as find and collaborate with developers of like mind and passion on this site.

Also, we fixed numerous bugs..  Nothing new here, we are always fixing bugs.

As always, feedback (either positive, constructive, or what the weather is like where you) is always welcomed!

Wednesday Apr 01, 2009

As you know by reading earlier posts on this blog, we have been working on improving the performance of our Web UI. The authentication user interaction is tightly integrated with rest of the Web UI to provide seamless experience in the rich Web UI. Combining this with the requirement to encrypt user names and passwords, we initially run the entire application under SSL. Ideally, we'd figure out a way to use SSL only for the parts of the communication that require encryption, and regular HTTP traffic for all other aspects of our application (e.g. search results, images, etc.) 

We are experimenting with a new feature that enables the web UI to interact with both SSL based web services and non-SSL based web services within the same browser page. This is one optimization that can help the web site to better scale with increasing traffic.

One technique is the use of iframes with fragment identifiers (text after the hash character in an URL, i.e. http://domain.com/page.html#tragment_identifier). This combination enables the web UI to establish a communication channel needed to pass small amount of information needed between iframes to interact with both SSL and non-SSL service points. For further explanation and a working example, please see this blog post for more information.

This use of fragment identifiers as a cross-domain communication channel is in direct overlap with popular practices for history management, which also use fragment identifiers to store page rendering state. Here is a sample page that shows how this feature works.

This leaves us with a potential situation when a page's rendering state can be wiped out whenever iframes use the same fragment identifiers to pass messages among them. In our particular use case, this is not a problem when the message actually leads to a new page state. If this is not the case, then it may lead to an inconsistent state between the representation in the URL and what's displayed on the page. One work around is to reset or synchronize the page state for such scenarios, which is a minor de-optimization that can generate repeat AJAX calls. This will be an interesting challenge to integrate conflicting features. Look for this performance improvement in an upcoming release

On Monday March 30th, 2009, we deployed build #69 of the Sun Software Library.  This blog entry details the additions we've made.

This release completes our work with Tag Clouds and Tag Controlled Vocabularies / Tag Folksonomies, which we started a few iterations back.  The approach we use is as follows:

  • We chose a set of tags that we display in our tag clouds.  While this list is fixed, we are open to any feedback to add or remove tags from those clouds.  A future release will allow personal tag clouds.
  • Any logged in user can tag entities in the library with those tags, causing them to be visible through the tag clouds.  This is the Tag Folksonomy aspect of the library.

This particular release enables users to quickly add tags that are in the tag clouds.  The new Web UI enables this as follows:

  • Clicking on the "add tag" displays the following (if the user is not logged on, they would be prompted to log in first, since only logged in users can add a tag):

    Add Tag

    Each of the dropdowns displayed corresponds to the list of tags in a particular tag cloud.  The user can select a tag from the list:

    Add Tag

    Then press the "add" button to actually add the tag:



    If the user wants to add a tag that is not in one of those lists, they can type it into the "other tag" field.  This field displays a list of matching tags corresponding to what is typed in.  The user is not limited to the list of tags displayed:



  • The editor can click on the "myLibrary" link on any of the Sun Software Library pages, click on "edit", then select the "Tags" tab, and add tags, as follows.  First the editor clicks on "myLibrary", then clicks on the "edit" button corresponding to the entry they want to edit:

    MyLibrary Add Tag

    The editor then selects the "Tags" tab:



    The editor then adds a tag, similar to the way tags are added in the detail page above:



    The editor can also add a custom tag:



    Finally, when the editor clicks the "add" button when the editor has selected the tags to add.  The tags selected are added to the tag list on the left side:



    Note that the tags are not actually published or saved until the "Publish" button is pressed:

That's pretty much it for this release..  We fixed a bunch of bugs, etc., but what else is new there?

Send us feedback!

This blog copyright 2009 by Ari Shamash