Safari Rough Cut release of zembly book

I have been working on a book describing how to use zembly to create the social web (think wiki for code targeted to social networks and other web 2.0 platforms like a blog), which was released via Safari Rough Cuts. There is still time to influence the book's content, so please use the feedback mechanism.

Posted at 05:18PM Sep 01, 2008 by Christopher Webster in General  |  Comments[0]

zembly

I have been working hard on zembly that began "perpetual beta" today. At zembly, we are trying to provide a service to easily create social network applications that run on social networking sites like Facebook and Meebo as well as hosting widgets.

One of our focal points is making invoking Restful services easy. There are several difficult (not necessarily in terms of technology) issues to deal with when consuming restful services including API key management, and perhaps one of the most challenging finding and determining relevant data providers.

We have also tried to remove the distinction of code and deployment space by developing and "deploying" applications in the browser. This is a similar model to how blogs or wikis have taken over what in the past was the tedious process of editing and uploading html and other content.

If you are interested in taking a look at zembly, please visit the site. This is currently in private beta but there are some invitations available  

Posted at 03:24PM Jun 06, 2008 by Christopher Webster in General  |  Comments[3]

Interested in an Internship

The team I am working on has several openings for interns. We are looking for people who want to build Facebook, MySpace, iPhone, and Meebo applications. If this sounds fun to you, please take a look at the job posting.

Posted at 09:12AM Apr 09, 2008 by Christopher Webster in General  | 

Working with the Facebook data API

I just finished working with the Facebook Data API to see how it compared with other storage services like Amazon S3. According to the documentation there is no intention to charge for normal usage of this service, so if you are building a facebook application it is worth considering this API. 

The Data API requires referencing objects by identity. All objects have intrinsic identity which is returned whenever an object is created. This can be referenced from FQL as '_id'. The Data API also supports the ability to reference an object using a user defined key (referenced in the documentation as a hash key). Just as in object oriented programming, objects can have properties which must be one of the following set of types (integer, string (255 char max), and text storage as a blob).

I used this API to allow associating tags with a user.  I started by defining the data model using the DataStoreAdmin link on application page. Using this tool, you can quickly create object and association types as well as execute FQL queries. I created two object types user (containing user id) and tag (containing the tag value) as well as the tagged association (a symmetric association between user and tag).

When a user is tagged the following operations are performed:

* Create a tag object by invoking data.setHashValue passing in the tag string for the hash key as well as the value property. This service returns the object id and allows the object to be referenced via hash key in addition to the object id. In this data model, invoking setHashValue is idempotent and thus this can be called every time a user is tagged.

* Create a user object by invoking data.setHashValue passing in the user id as the hash key and the id property.

* Create an association using the data.setAssociation service to link the user object and the tag object. The id's returned by setHashValue should be used to collect the id's.

When the tags for a user are displayed, the tagged association can be queried, using fql.query, to determine the set of tags applied to the user. The query for extracting the tags would roughly be:

SELECT value FROM app.tag WHERE _id IN
   (SELECT tag FROM app.tagged WHERE user = tagged_user)

The storage mechanism makes this data model pretty straightforward. This hash key mechanism can store properties of non scalar data such as JSON. This supports the ability to use the getHashKey method to retrieve more than a single property. There is also an interesting feature of hashkey, data.incHashKey, which allows atomically modifying an integer property value.

 The example above could be extended to support a tag cloud, by counting how many instances of the tag have been added. One way to achieve this is by adding a count object with a single count integer property. The count object type would have an association with both the user and tag objects to have a unique count for each user. The count would be obtained through a query over both associations. Another implementation technique would be to use a synthetic hash key and a unique tag object for each user. The tag object would be extended with the count property. The tag object's hash key would combine the user id and the tag name. The count property would be incremented each time the user is tagged.

Observations

• The API would be smaller if only hash keys were required for each object. The object id is somewhat difficult to work with and typically domain data provides unique identifiers.
  
• The batch API is required for any significant use of this API, as a single logical operation will translate to multiple service invocations. It would be nice to have more support directly in the data API.
• A lot of common use cases could be supported by extending the increment hash key capability to any object property. This could be done by passing the last version id and the current update and rejecting the change if an intervening change has occurred. In relational databases, this is implemented using an optimistic locking technique such as a version column.
  
• This technology is worth evaluating if you are working with a viral facebook application.
  

Posted at 06:10PM Mar 30, 2008 by Christopher Webster in General  | 

Caching static resources in glassfish

I am working on a project which has a fixed daily deployment schedule, Monday through Friday there is a 9:45 deployment. I wanted to take advantage of the known deployment schedule to incorporate Cache control directives when serving static resources. Currently, Glassfish is serving the static resources so I wrote a simple Filter which I applied to the static resource patterns. The filter is instantiated when the web module loads, so the next deployment serves as the appropriate caching period (the period will vary based on the day, the Friday deployment will be cached over the weekend). 

Glassfish (as well as other servers like Apache) will set the Last Modified and ETag headers. These headers still require the client to issue an HTTP request for the resource; however, the server may respond with a 304 status code which indicates the content was not modified since the given date and etag (a mechanism for detecting change in the contents of the resource, like a hashcode). This is accomplished by the client sending the If-Modified-Since header along with the If-None-Match header to let the server know the date and "signature" of the file currently cached. The server will return a 304 if the response would return the same resource which saves the network transmission time for the resource.

This approach doesn't solve the problem where browsers generally only request two resources from the same domain in parallel, so if there are hundreds of resources on a page the overhead of determining whether the resource was current could impact the page load and hence the user experience. Instead I wanted to use the regular deployment cycle to reduce the number of requests required to determine whether resources are up to date.

The filter sets both the HTTP 1.0 Expires header as well as the HTTP 1.1 Cache Control Directive, to ensure both protocols are covered. If you are using apache to server static (or dynamic) resources you can find a good resource for setting up apache modules to do something similar on websiteoptimization.com. Glassfish has admin console capabilities for configuring caching which should be evaluated before doing everything in code.

If you want to give it a try in a web module, you will need to edit web.xml to add a filter and a filter mapping:

          <filter>
        <filter-name>ExpiresFilter</filter-name>
        <filter-class>server.ExpiresFilter</filter-class>
    </filter>
    <filter-mapping>
        <filter-name>ExpiresFilter</filter-name>
        <url-pattern>/images/*</url-pattern> <!-- these patterns should match cached resources -->
        <url-pattern>/resources/*</url-pattern>
        <dispatcher>REQUEST</dispatcher>
        <dispatcher>FORWARD</dispatcher>
    </filter-mapping>

Add the filter class to the web module along with adjusting the appropriate timeout.

Posted at 10:17AM Jan 21, 2008 by Christopher Webster in Java  |  Comments[1]

Interesting Site for a collection of web links

Posted at 09:45PM Jan 05, 2008 by Christopher Webster in General  |  Comments[1]

Dapper Camp

Dapper is an interesting software as a service website that serves as a transformation engine for the content of a website. The content can be converted to a variety of different formats including RSS, XML and JSON. One of the common use cases for Dapper is to select some subset of a website's content, which is certainly interesting. However, a more interesting use case for "Web Service" type applications is the ability to create an external API for web sites which do not have API's defined. These API's are constructed using the Dapper interface which allows publishing a RESTful service that can then be consumed by service side mashup services. There is a free dapper camp in San Francisco in February where you can learn more about this technology.

In addition to the dapper camp, a contest to build a NetBeans plugin is underway. If you are considering enhancing the NetBeans REST or other web technology support this provides an interesting incentive.
 

Posted at 02:39PM Jan 02, 2008 by Christopher Webster in General  | 

NetBeans YouTube Videos

In addition to NetBeans.tv, Sherry has been busy working on some You Tube Videos for NetBeans. There are several recent videos, one of which I participated in making. You can find a videos on:

• RESTful services
• XML Schema Support
• SOA Support 

Posted at 06:41PM Oct 10, 2007 by Christopher Webster in Netbeans  | 

NetBeans 6 Web Service Designer

NetBeans has introduced a Web Service designer, as you may have seen in Geertjan's blog. In addition to the ability to graphically view and modify operations for implementation first web services (WSDL is generated based on the JAXWS annotations), the web service designer provides the ability to work incrementally starting from a WSDL document. This represents a significant enhancement to the development experience as the WSDL document can become a first class source artifact, while the developer experience remains the same as the Web service designer provides the same implementation
 view. The add operation feature has also been extended to support the direct addition of XML schema. Direct addition of schema elements will perform an on demand generation of JAXB to provide a Java binding.

 Having the WSDL generated as part of deployment presents makes it difficult for composite application assembly tools, such as  the NetBeans CASA editor, to take advantage of the abstract WSDL binding capability of the openESB runtime. A composite application provides code composition based on well defined WSDL based interfaces. Since all the modules are deployed together and each WSDL interface is exposed on the service bus, developers can rely on bus endpoints instead of the concrete endpoints exposed in WSDL, thus achieving loose coupling as only the abstract WSDL is needed. The WSDL document can also be modified using the NetBeans WSDL editor to enhance the generated information such as adding documentation.

You will also notice the ability to see sample input and output messages. This is useful when working from Java types to see  how changes to the structure of Java classes will affect the XML messages.

This new feature is worth a look, you can find this in NetBeans 6.  

Posted at 08:04AM Aug 25, 2007 by Christopher Webster in Netbeans  |  Comments[1]

Unit Testing HTTP Calls

I have been working with RESTful web services and had the need to verify the HTTP messages I was sending were appropriate. Since I am doing this work using Java SE 6, I was able to take advantage of the httpserver package. This package provides a lightweight HTTP server and is used for things like publishing web services and displaying memory graphs based on jmap.

The code to take advantage of this is easy. As you can see below, my strategy was to get an arbitrary open port on the local machine, bind a test handler to a context where the message is directed, and then verify what was sent matches what was expected.

 InetSocketAddress address = new InetSocketAddress(0); // get free port on local machine
HttpServer server =HttpServer.create(address, 1); // create a server on the arbitrary port 
int portNumber = server.getAddress().getPort(); // get the port where this is bound for the invocation
        
MockHttpHandler handler = new MockHttpHandler(); // create a mock handler to capture the HTTP input
try {
    server.start(); // start the server
    server.createContext("/test",handler); // create a context with the mock handler
 ...
        } finally {
            server.stop(0); // stop the server after the test is complete
        }

Posted at 11:57AM Aug 21, 2007 by Christopher Webster in Java  | 

Free Open Source Desktop project management tool

Posted at 03:55PM Aug 12, 2007 by Christopher Webster in General  |  Comments[2]

Working with Facebook Web APIs

I have recently been working with the Facebook APIs. I wanted to see how to incorporate Facebook data in a widget. By widget I am referring to reusable UI behavior which is driven by a set of services available via http. In contrast to the typical Facebook enabled web application, I wanted investigate the feasibility of using Facebook services as part of a mashup. Specifically, I wanted to ensure that without full page control (iframe embedding), it is possible to interact with Facebook services.

Here is what I did:

1. From my Facebook account, I added the developer application. The developer application lets you create new Facebook applications. Creating a Facebook application generates a public and private key set which is required for invoking the APIs.
   
2.  In order to start calling the API's to access Facebook profile information, authentication must be performed. The Facebook platform authentication mechanism is similar to the OpenId mechanism, whereby the Facebook platform actually performs the authentication and provides user information to the application. Since the target use case is to embed the widget within a Mashup and hence the reusable widget would not know the semantics of the page, the callback mechanism for web applications (after authentication on facebook, facebook redirects the browser to the provided URL) is not really appropriate. Facebook also provides a desktop authentication mechanism where a token can be generated, once the user is authentication (perhaps by launching a dialog or another browser window) the token is activated. The application can then obtains a session lease where the user id, the session id, the expiration time, and a session secret are provided. The widget can use the approach to generate the token before launching a facebook login. One problem with this approach is the exact order of obtain token, user authentication, get session must be preserved. If getSession is called before the user actually authenticates, then the token is invalidated and get session returns an error. This is a normal challenge with distributed authentication. The message signing is similar to other web APIs such as flickr, so that will be second nature to most people.
   
3. Following, this I used the Facebook Query Language to start extracting data. This allows flexible queries (there are some limitations where the query must contain an index and there are no joins allowed) on different data sets.
   

Posted at 09:02AM Aug 10, 2007 by Christopher Webster in General  |  Comments[1]

JavaScript code coverage tool

Posted at 11:46AM Aug 08, 2007 by Christopher Webster in General  |  Comments[2]

Presentations about scalable web architecture

Posted at 06:58AM Jul 27, 2007 by Christopher Webster in General  | 

Workaround to get more information for the JPA exception: During synchronization a new object was found through a relationship that was not marked cascade PERSIST.

I was getting a bit frustrated dealing with the JPA exception During synchronization a new object was found through a relationship that was not marked cascade PERSIST. The issue is that the message doesn't display the object in question. I wanted to grab the code and fix this issue, so here is what I did

• download the persistence source from https://glassfish.dev.java.net/downloads/persistence/JavaPersistence.html. I am working with V1 so I used that version.
  
• I copied the class I wanted to modify to my unit test area using the same package structure as in the persistence jar file. I am using NetBeans for unit testing so I adjusted the unit test classpath to put the test classpath ahead of the toplink essentials jar file which effectively replaces the class.
  
• Looks like this is just a bug where the IllegalStateException thrown in RepeatableUnitOfWork line 115 (as we all know and love) is passing the object, so all that needs to be done is to fix the error message. I found ExceptionLocalizationResource where the error messages are stored and added a parameter to the new_object_found_during_comment message. I changed it to: "During synchronization a new object was found through a relationship that was not marked cascade PERSIST. {0}." This will provide the output of the toString method for the object where EntityManager.persist was not invoked. Since the toString is in the user space, this allows me to add any additional information to help figure out where to add CascadeType or call persist.

After doing this, additional information on which class (and the toString) is now contained within the exception message.

The glassfish tracking issue has been filed as https://glassfish.dev.java.net/issues/show_bug.cgi?id=3254 

Posted at 06:51PM Jun 16, 2007 by Christopher Webster in Java  |