e hënë janar 31, 2005

Cool, Danny Ayers discovered our little java-syndication group and wrote a thoughtful blog entry with a proposal for something that we want to solve: Syndication model convergence and RDF.

2 weeks ago Alejandro and I (from ROME) went to San Francisco to have lunch with Kevin Burton and Brad Neuberg from Feedparser and discussed potential areas of collaboration, and our joint submission at JavaOne. One of the ideas that came up (from Brad if I remember) was to build ROME-like beans on top of a Feedparser-like event driven API. We also agreed that the first thing we could collaborate on was on specifications for the mapping. After lunch I suggested that we could eventually move the mapping logic (in ROME terms, how you map a format specific WireFeed to an abstract SyndFeed) to a declarative representation, so that instead of having written specs we could generate code. But I didn't elaborate much more: I was not sure it would be necessary if we build one parser on top of the other. Most of the logic would migrate in feedparser, to generate the Synd level event, and ROME would just listen to these events to build beans.

What I had in the back of my mind when suggesting that was linked to my recent dabbings in Ruby. A quick look at the Ruby Application Archive showed 3 syndication libraries:

• ruby-rss, antique, from 2001, probably only good for RSS 0.91
• rerss, last updated in 2003, good for RSS 1.0 and 2.0.
• rss, last updated in december, this seems to be the live one. It handlles RSS 1.0, 0.9x/2.0 with validation (or non validation), but not sure about generation, no Atom, and not sure about the object model.

I had not started thinking about which format to use for this description, but Henry Story's work on mapping Atom to OWL (continuing the work Danny had started) leapt to my mind (he presented it at the october 21st ossgtp in Paris). Danny fills in the blanks in his post! I wanted to spend a bit more time looking at alternative formats and tools for this description (RelaxNG, XMLSchema?) before starting on that, but this application seems to me like an excellent real world use case of RDF/OWL.

e enjte janar 27, 2005

Like all java geeks I've been learning Ruby recently: it's always refreshing to learn new ways of thinking about things. In Ruby the main way to think about programming is in terms of iterators and blocks.

A few references for my friend Erwan:

• Programming Ruby aka the Pickaxe, by our friends the pragmatic programmers. Dense, terse, fun, as all their books. It's the classic for Ruby. I had a similar pleasure reading it as when I read Programming Perl many years ago (except that I never finished Programming Perl, and very quickly did not have any occasion anymore to practice what I had learnt). Start with the Pickaxe.
• Ruby homepage: the Ruby official homepage, Ruby Application Archive, to search for Gems, and RubyForge listing Ruby projects.
• Rolling with Ruby on Rails at O'Reilly, by Curt Hibbs, from last week: a very windowish tutorial (pages of screenshots to explain what would be done with a few commandlines on a Unix system) but it explains very clearly the basics of the Rails Ruby webapp MVC framework (I guess O'Reilly's target audience is changing these days:-).
• JRuby: a ruby interpreter in java. Get the best of both worlds, Ruby for the higher level logic, and all the power of your favorite java libraries. Or embed Ruby in your favorite app server. They released a 0.8 version 4 days ago, with full support for Ruby 1.8.x grammar.
• Thanks to James Britt for pointing out in a comment the very good rubydoc weblog, a blog about new Ruby documentation.

I have a few pet project ideas that I want to do in Ruby: let's see how pleasurable it is to code with!

Pourquoi Google s'interesse a Java et Mozilla ? is a very good analysis of the recent Google hiring moves for java and Mozilla engineers. It's in french, but Google is your friend and here's how they would translate it (thanks to the anonymous commenter who posted this link).

Ludovic expresses in very clear terms what I was thinking since a while. Ludo and I were at Netscape together, when the strategy was to make the OS irrelevant by having the browser be the platform for most applications. I guess Netscape was too early in this game, but Google has much more chances to make it work now.

The only point I would add to Ludo's analysis is that I think Google will add java on the client as well, in order to be able to add functionalities to their apps on the fly. If you've ever tried creating an app in Mozilla, you know that all the deep functionality is written in C code, wrapped in XPCOM bindings and accessed from the UI code through javascript. So if you want to add serious services to Mozilla you need to deploy new C libraries with an XPCOM binding. I think Adam Bosworth is going to implement the client side caching/synchronization framework he was talking about when at BEA in their Firefox based browser, but that they will probably add a VM in there, with java-XPCOM bindings (does someone remember the now defunct blackwood project, maybe Google is going to finish the work:-) that will let them provision jars with new functionalities from the network. Then the UI will be written in XUL-javascript. This would be an ironic revival of fortune for java on the client: java on the client, but not for the UI stuff:-)

These are just educated guesses and may prove completely wrong. We'll see. But if you think about it, this was the architecture that Kevin Burton, now chief architect and founder at Rojo, had chosen 2 years ago for his well named Desktop news aggregator NewsMonster.

e martë janar 25, 2005

Today Sun started the open sourcing of Solaris by releasing the source code for DTrace. I'm a java guy, with a little experience in C and C++, but not familiar at all with UNIX kernel engineering, so I guess this code will be way off my head. But I decided to download it and take a quick look at it.

The OpenSolaris site has a neat simple design, with a list of blogs on the right, but no RSS feed yet. I hope they'll add that soon.

The download is quick, and I was very happily surprised that you don't have any form to fill out, or email/password to remember to get to it: click on the link and download.

The code?

find . -name '*.c' | xargs wc
...
 82678  272892 2111307 total

and

find . -name '*.[c|h]' | xargs wc
...
   91380  322081 2468438 total

So it's 82 kloc (Kilo Lines Of Code) of C code, plus 9 kloc of headers. I guess I need to finish Diomidis Spinellis' excellent Code Reading: The Open Source Perspective which sits in my pile since a while before even starting looking at it!

For all you C people: happy code reading.

Todd Fast explains HOWTO: Adding Technorati Cosmos Links to a Roller Blog. A good entry, but since he is not aware of all the possibilities offered by velocity macros in Roller templates he has to use javascript to encode the url of the post, which makes including these links more complicated than it should be.

Roller macros are extremely well documented at http://www.rollerweblogger.org/velocidocs/index.html, but I could not find any documentation for the $utilities package (or whatever they name it in velocity). The best way to discover what you can do in velocity macros inside of the Roller context is to read the code from roller macro packages (in the documentation the code is included at the end, which is very convenient). The only problem with Roller velocity templates is that you cannot use many of the excellent velocity tools, since they are not installed in Roller. I tried installing one of these on my own Roller server and found out that you can't do it without modifying Roller code because Roller populates the velocity context in some java code in a different way than the standard velocity servlet.

In order to encode a url you can use the $utilities.encode function in your template. $utilities has a set of useful functions that are used by the Roller templates tehmselves. If you are curious for more, get the Roller code and find all .vm files: they are worth a read if you plan to do more than boilerplates templates. roller.vm is a good read.

So I would replace Todd's instructions by this code:

#set ($permalink = "$absBaseURL$entry.permaLink")
#set ($encodedpermalink = $utilities.encode($permalink))
<a href='http://www.technorati.com/cosmos/search.html?rank=&url=${encodedpermalink}'>
<img align='top' style='border:0px' alt='Technorati cosmos' src='${absBaseURL}/resources/${userName}/cosmos.gif'/></a>

For those interested in James Todd's MyJxta experimentations, I also have similar code in my template, coupled with a servlet on my server to autogenerate a jnlp file for MyJxta starting a chat room for each item in your blog.

#set ($jxtadescription = $utilities.encode("Chat room for ${entry.title}"))
#set ($jnlplink = "http://blogs.chanezon.com/roller.jnlp?name=${userName}&id=${encodedpermalink}&description=${jxtadescription}")
<a href="$jnlplink">Chat about this entry</a>

Feel free to experiment with it, but I will describe this further when it will work as advertised.

e mërkurë janar 12, 2005

I got some pretty good feedback after my post Java Syndication Babel: let's paint the picture together!, mainly from the Rojo/Feedparser team.

Adewale Oshineye who uses Informa and makes the desktop aggregator Aggrevator is against a JSR, but in favor of sharing tests, and merging projects. Good feedback: when we started Rome we asked Mark Pilgrim if we could use his Universal feedparser unit tests for Rome. We ended up not using them because they were structured bizarrely, but we reused his idea of starting a web server and serving feeds in the fetcher unit tests. I agree that a common set of unit tests would be excellent. I don't push for a JSR, I just want to start a conversation: we'll see as we go what form of collaboration are best, and wether a JSR is worth it or not.

Then the Rojo/Feedparser team, Brad Neuberg and Kevin Burton posted some good comments. I agree with Kevin that we don't necessarily need a JSR and should start collaborating now. From Kevin and Brad it seems like what we should work on first is defining a common set of Unit Tests, and a way to get a DOM-Rome-like API on top of their SAX-like API.

Following Kevin's suggestion I created a Yahoo Group to start working on that, and we can use the Rome wiki to write docs.

• java-syndication Group home page
• Group email: java-syndication@yahoogroups.com
• java-syndication wiki

Feel free to register, and I hope the Informa team will join us as well.

e martë janar 11, 2005

Alejandro released ROME 0.5 yesterday. See the wiki for details. The big news is that it is our first beta release: we consider that the APIs have stabilized enough that we can call it a beta.

e hënë janar 10, 2005

We're nearing a beta for ROME, so it's finally time to get serious:-) What would an open source project be without a preso and a T-shirt?

I finally posted the presentation I gave at a Sun internal open source summit this summer, plus the T-shirt I designed for the event.

The presentation is on the ROME Wiki, in PDF or Staroffice form: feel free to reuse it if you want to present ROME, and enhance it. It was designed for ROME 0.4, so it may need a bit of updating.

The T-shirt is just a quick fun image I hacked before taking the plane last summer: a project isn't real until you get the T-shirt:-) You can use the image at the top of this blog entry and iron it yourself, which I did, or use my CafePress shop to get it.

When we started the ROME (Rss and atOM utilitiEs) project last may, we had done a preliminary review of existing syndication libraries in java, and none of them suited us: we try to avoid the Not Invented Here syndrom as much as we can:-)

The other day Kevin Burton commented about our Xtech 2005 article submission about Rome that he would submit a paper as well about Jakarta Feedparser. We were not aware of this project when we started Rome: it's hidden in the bowels of the Jakarta Commons Sandbox (I think I did a quick search on apache rss at that time and found nothing meaningful: Google makes you lazy!). Too bad because it looks pretty good. And this is not surprising because Kevin is the guy who created this very innovative aggregator called NewsMonster a few years ago (what i thought about it in my very old weblog, and my old weblog), and started a more promising online aggregator implementing the same ideas (collaborative filtering, reputation metrics, tagging) a few months ago: Rojo. According to the project's mailing list, Feedparser is based on NewsMonster's libs, and is what they use at Rojo.

A cursory look at Feedparser makes me think that our architectures are quite different: they seem to have an event based approach, defining SAX events at a higher semantic level (like SyndFeed and SyndEntries in ROME, but based on events at parse time), while in ROME we handle you Java objects once the parsing is over (more like a DOM-based approach). But even if the architectural approaches are different, they have a few goodies that could be useful for us, such as nice autodiscovery classes to guess where the feed associated to a page is, based on the system used to generate the feed (if people omit the RSS or Atom autodiscovery link in the template). And they could benefit from a few things in ROME, such as the fetcher, and Modules.

Also in the few libraries that we studied before deciding to create ROME, only Informa seems alive and kicking.

Joseph Ottinger who created RSSLibJ switched to ROME when we shipped 0.1, and is now part of the ROME developers, with very active contributions to the design debates we have in the mailing list. And Jason Bell decided to stop development on RSSLibJ recently and use ROME.

If you're doing syndication in Java today, you have the choice among these 3 libraries. Kevin's comment arrived at a good time: Alejandro and I are preparing a submission for a technical talk at JavaOne this year. And we're also beginning to think that for all future work on Syndication utilities (extensions, applications, rdbms mapping, etc...) it would be really cool to be able to leverage a common java representation of feeds and entries, instead of having your development tied to the library you have chosen... you see where I'm going don't you?

What I propose is to submit a joint paper to JavaOne this year, where the 3 teams will present their project, and to work together to determine where we are different, and similar. I think having a clear picture of what the various libraries' strength, weaknesses and designs are would be much more worthwhile for attendees than a single one (and I would be the first interested in such a presentation).

Then take advantage of JavaOne to host an open meeting to discuss the potential convergence points of our projects, and wether it is worth standardizing them in a JSR, something like a Java Syndication API. One of the reasons why I think such a JSR could be useful is that with the new Atom publishing API soon published, it would be good to have a common java API to get and set feeds. Dave Johnson plans to add a new ROME subproject for Atom publishing where yet a few Interfaces will be defined for a blog, blogsite, BlogEntry. It would be nice if all of our libraries would agree on common java interfaces for this kind of data so that they are interoperable.

Kevin, Niko, Pito, Joseph, DaveJ, Alejandro, what do you guys think? If you all agree, the deadline for the JavaOne call for paper in January 31st, so we have 2 weeks to produce a common outline.

e enjte janar 06, 2005

This post is not about my weblog skin, I use a default theme coming with Roller, but about how tools and culture interact to affect how we collaborate, and more precisely how our culture, personal or corporate, affects how we use weblogs, mailing lists and wikis to collaborate.

At Sun we've historically been mailing lists people: we live in our mail client and mailing lists are the primary form of informal knowledge sharing. At Netscape we were web page people: everyone had their own web server internally, and an external web site at http://people.netscape.com (nostalgia, I remember the rotating cube logo and the old black and white Andersen Consulting mugshot with a tie:-).

I had trouble adapting to Sun culture when I switched to Sun in 2000 (doing the same job in the same team, as part of the iPlanet joint venture): I was used to organize all my docs my own way, on my own server, and had to switch to Sun habits where you usually have a big common server where all formalized engineering docs are organized, and most active knowledge sharing happens in mailing lists.

The difference in web publishing was not so much in the technology used, as in the culture: Netscape was all about individual personality, Sun about conformance to a process. At Sun you showed your personality in emails, at netscape in web pages.

There are drawbacks to both approaches: the Netscape intranet was a mishmash of unorganized content, and you had to know which people's page to look for the right information. This was possible for a 2000 people company it was at that time, but was beginning to show scaling issues when we were bought by AOL. At Sun the drawback, for me at least, is that you have to follow a process, which changes regularly, use templates, and my personality does not fit very well in processes (I liked that quote form the Nicolas Cage character in David Lynch's Wild at heart: "This snake skin jacket symbolizes my individuality and belief in personal freedom!")

I continued to publish many of my docs my own way, either on my own server, or in me personal account on the internal server. But it does not work very well: my server machine was moved and stopped working very often, rendering my documents unreachable, and Sun IT changed my UNIX login name so all my urls went 404.

Today I use a mix of internal blog, external blog, internal and external mailing list, internal wiki, and internal project publishing site to share my docs and thoughts. The advantage of blogs over personally managed web sites is that the blog server (as externally we use Roller internally) is managed by Dave (thanks Dave), so it's always up, but it lets me personalize my page and posts. The best of both worlds.

How do blogs, internal or external compare to mailing lists: there's a new balance to find between both and we're experimenting with these modes of collaboration since a few months. Most of the informal knowledge at Sun was traditionnally shared through mailing lists. Since a few years people have been experimenting with internal wikis as well, and these have started really picking up steam in the past 6 months.

The problem with mailing lists is not technical but behavioral. We have some good web based search for mailing list answers so you could say that a post to an internal weblog and a post to a mailing list are equivalent: they both can be reached through a url. Technically it's true, but the behavior when answering a question in a mailing list and writing a weblog post is different. For mailing lists the main mode of access is through a mail client, and information is considered transcient, because difficult to search. Better desktop search tools, such as Google, Microsoft or Copernic maybe will change that perception. The excellent web based gmail which puts search at the center of email management will probably also change this, but it's not available for intranets yet. So when you answer an email you're in a mode, or mood, where you create something for the purpose of this only interaction, and where reuse is difficult. To the contrary, when posting to your weblog you think that you publish something that will live forever at the end of a url, so you try to make it more general, less conversational, more long lived.

This attempt at behavioral psychology in the wild (*) is not very scientific and I look forward to see real sociologists or psychologists analyze how our cognitive behaviors are impacted by the tools we used, but I think it's a useful distinction to keep in mind when analyzing how people collaborate with email and weblogs.

Simon Phipps had an interesting post about flows of conversations between wikis, weblogs and email. I'm not sure wether his picture is right, I think it is too early to say: we're just starting experimenting with these various tools and I expect many surprises down the road as the tools evolve (podcasting will soon add an extra dimension to this landscape, adding voicemail in the picture). These are good times for experimentation.

At my level, I decided to post more to my weblogs, internal and external. I've been pretty active in a few internal Sun mailing lists in the past few years, especially in Portal land, which is my main specialty. But I never posted anything related to Portal on my public weblog. I'm going to start blogging about Portal more, instead of answering in the mailing list: I'll just answer with links to my weblog, internal or external depending of the confidentiality of the information. Let's see where this leads me.

I look forward to your comments about this topic, to which I will come back after more experimentation.

Today I got the following question from Rick Evans, our course developer. I'll answer in my weblog in order to start sharing this kind of information with the community, and hoping to trigger interesting comments. It has been a while I wanted to talk more about JES Portal in this weblog and this will be a good start.

I am curious what "the professionals" consider the best practice when creating a new portal in the field.

Or more importantly, what best practices do you want your customers to be taught?

-------
The following alternatives occur to me:

1. Customize/modify the Sample Portal itself.
2. Do not install the sample portal during Java ES installation, and create a new portal from scratch.
3. Create a new portal and install it as a new additional top-level portal.
4. Continue to use a suborganization for the new portal - hey, it works!
5. Other alternatives?

I appreciate any and all comments on what you recommend and the advantages/disadvantages of each approach.

Best regards,

Rick Evans
Portal Server Course Developer
"Good training makes everyone's job easier." 

I'm a Portal architect and developer, so usually I either work on the guts of JES Portal, or develop new Portlets to be included in the release, or used by Professional Services or internally. So I'm not best qualify to answer this one, but I'll give my perspective. Professional Services folks such as Ulf Feger, Marc Perret or Jeremie Van Renterghem will probably have much better answers.

In order to develop a new Portlet I usually use the sample Portal as is, and create a test role. I then create a small display Profile fragment to add that portlet to the role, and create a channel using this portlet in one of the containers of the sample Portal. I then use dpadmin to upload the fragment to a running vanilla Portal for my role.

Thta is the easiest way to develop a Portlet, or Provider, with JES. It also has the advantage that when people who use your Portlet get your code, they have a small easy to understand display profile fragment to work with in order to test it in their staging Portal before real deployment.

On one project I also developed a container, ie something that lays out portlets on a page. For this I used the same methodology.

But I never developed a full Portal for a customer, but my take on this kind of project is to follow the good old XP (Extreme Programming) practice of working in small iterations getting at any time something that works. This rules out trying to develop a new Portal from scratch (which means creating a brand new display profile). Instead, reuse as much as you can of the sample Portal.

But which kind of reuse? Inheritance, using a suborg and specializing the display profile for it, or code level reuse, modifying the sample display profile. I'd say it depends on the scale of what you're doing: if you develop something really different from the sample (using custom containers for example) I'd say code level is best, else inheritance is best.

Anyway what I'd recommend is a bit different from the options you list. I'd leave the sample Portal as is, as a baseline and test that your system works fine. I would create a new Portal copying the sample one. Then start modifying the display profile, or create suborgs depending on the needs. But once more I may not be best qualify to answer this question: comments from field practitioners are welcome: Ulf, marc, Jeremie, when do you start your weblog?

e mërkurë janar 05, 2005

I just submitted my proposal for the XTech 2005 conference (hutty up, the deadline is in 2 days! Thnaks to Eduardo Gutentag for warning me): ROME: ESCAPE syndication hell, a java developer perspective.

I hope it will be accepted: we'll release a beta of ROME this week and it's time to make some noise about ROME!. Here's the abstract, comments welcome.

The XTech 2005 Call for Paper application had a bug, and the site administrator fixed it in less than 2 hours after I notified him by email: Kudos to him, what a great reactivity! I wonder what software powers their site.

ROME: Rss and atOM utilitiEs in java is an open source project aimed at developing a set of Atom/RSS Java utilities that make it easy to work in Java with most syndication formats. ROME and its various subprojects will be presented.

1. What is ROME

ROME: Rss and atOM utilitiEs in java is an open source project aimed at developing a set of Atom/RSS Java utilities that make it easy to work in Java with most syndication formats. ROME and its various subprojects will be presented.

2. the ROME library

The ROME project itself includes a set of parsers and generators for the various flavors of syndication feeds, as well as converters to convert from one format to another. The parsers can give you back Java objects that are either specific for the format you want to work with, or a generic normalized SyndFeed class that lets you work on with the data without bothering about the incoming or outgoing feed type.

Today it accepts all flavors of RSS (0.90, 0.91 Netscape, 0.91 Userland, 0.92, 0.93, 0.94, 1.0 and 2.0) and Atom 0.3 feeds. ROME 0.5 released in january 2005 is the first version of ROME marked as beta.

Our requirements for ROME are to ESCAPE from Syndication Feeds Hell. In order to allow that the library must be: E asy to use, S imple, C omplete, A bstract, P owerful, E xtensible.

3. ROME fetcher

The ROME fetcher subproject implements all the nitty gritty details involved in fetching feeds over http, including encoding detection, which required some interpretation of various RFC and XML specifications.

4. ROME Modules

We expect embedding of additional namespaced data in syndication payload to be an important trend in the next few years and designed the ROME architecture to allow for this: ROME has an extensible plugin architecture.which allows developers to create module handlers for the various extensions allowed by the syndication formats. Today ROME handles Dublin Core and Synd modules from the RSS 1.0 specification. We plan to implement all modules defined in the various specifications.

6. ROME plans and usage

Our most immediate plan is to create a new subproject called the ROME weblog client library. We expect Dave Johnson to contribute code for this. It will contain an implementation of the ATOM publishing protocol, as well as the older Meta Weblog and Blogger protocols. This will let ROME users publish items that have been generated using ROME to servers which understand these protocols.

Another potential development angle is to include a RDF library in ROME, in order to make it useful for Semantic Web applications, following Henry Story and Danny Ayer's work on building an Atom OWL ontology.

A list of people and projects using ROME: SnipSnap, Roller...

e hënë janar 03, 2005

When reading Perhaps now is the time to lay RSSLibJ to rest I felt a bit sad for Jason to have to drop his open source project, but glad that he chose Rome to generate feeds. We started Rome a few month ago, initially to create a single java library to deal with syndication feeds for Sun products. Then we decided that this library could be useful to many people outside of Sun, and thought that it would better be maintained by a community of developers than by just Sun employees (following Bill Joy's old quote that the most clever people do not necessarily work for your company), and made it open source as a java.net project.

This was an excellent decision: we've had 5 non Sun developers joining us and developing new subprojects, the project is very successful, and is nearing a 0.5 release where we will probably move it to a beta status (beta as in "the APIs seem stable enough now").

Jason if you feel the itch to code some utilities for syndication, just join the fun! Else your suggestions of how to make Rome better suit your syndication needs are welcome.

calendar

search

navigation

links

rss feeds

referers

Today's Page Hits: 51