Colm Smyth's Weblog

Lego -> Social and "Fictional" Software

When I read Paul Lamere's blog 27 Stages of Lego Sorting, it reminded me of Douglas Coupland's wonderfully insightful book Microserfs which is about a group of developers (some of whom had major Lego phases, one of whom goes on to create a Lego masterpiece ;) who form a kind of geek commune around a company to develop a software application called "Oop!". The idea of "Oop!" was to provide a rich kind of virtual Lego set, with the ability to use, build and share smart mobile & reactive components (a bit like Lego Mindstorms), which could act like doors or lifts or game characters or Rubik's cubes or vehicles, and so on. Ok, pretty interesting idea.

But that started me thinking; that's not the first time I've seen a software application described in a novel. Douglas Adams in Dirk Gently's Holistic Detective Agency also outlines a rather amazing piece of software which is best described by Douglas (2) himself:

'Well, Gordon assigned me to write a major piece of software for the Apple Macintosh. Financial spreadsheet, accounting, that sort of thing, powerful, easy to use, lots of graphics. I asked him exactly what he wanted in it, and he just said, "Everything. I want the top piece of all-singing, all-dancing business software for that machine." And being of a slightly whimsical turn of mind I took him literally.

'You see, a pattern of numbers can represent anything you like, can be used to map any surface, or modulate any dynamic process -- and so on. And any set of company accounts are, in the end, just a pattern of numbers. So I sat down and wrote a program that'll take those numbers and do what you like with them. If you just want a bar graph it'll do them as a bar graph, if you want them as a pie chart or scatter graph it'll do them as a pie chart or scatter graph. If you want dancing girls jumping out of the piechart in order to distract attention from the figures the pie chart actually represents, then the program will do that as well. Or you can turn your figures into, for instance, a flock of seagulls, and the formation they fly in and the way in which the wings of each gull beat will be determined by the performance of each division of your company. Great for producing animated corporate logos that actually mean something. 'But the silliest feature of all was that if you wanted your company accounts represented as a piece of music, it could do that as well. Well, I thought it was silly. The corporate world went bananas over it.'

Reg regarded him solemnly from over a piece of carrot poised delicately on his fork in front of him, but did not interrupt.

'You see, any aspect of a piece of music can be expressed as a sequence or pattern of numbers,' enthused Richard. 'Numbers can express the pitch of notes, the length of notes, patterns of pitches and lengths.' 'You mean tunes,' said Reg. The carrot had not moved yet. Richard grinned. 'Tunes would be a very good word for it. I must remember that.'

(aside: I think Douglas Adams had several important insights into software, one of which is quoted at the end of an interesting article Autistic Social Software (not really about autism, but about the ADD of software products and/or developers who don't want to try to understand the social implications of networked communication and collaboration; if you're into groupware though, my single favourite groupware article is one that I read 10 years ago - What do groups need? A proposed set of generic groupware requirements - it offers an excellent synthesis and framework for evaluating groupware requirements; gosh, back then I used e-mail to request a copy from one of the authors Munir Mandviwalla; he kindly sent me a hardcopy which I still re-read occasionally - great stuff, and now you can get it on the ACM portal; both of these articles only go to show that creating truly new forms of groupware is very hard)

I bet that's just the tip of the iceberg though in terms of creative software ideas in fiction; anyone know any others?

(2004-09-15 14:11:33.0) Permalink

Physics, Communities and The Tao of Democracy

After reading redbeetle's blog about What the Bleep Do We Know?, I think I've found another movie I need to go and see (I already added the director's cut of Donnie Darko on my list when it came out a couple of weeks ago; I've been curious since I managed to catch 80% of it on the tail end of a plane trip when it first came out ;). However, the combination of physics and spirituality is not that unusual; Fritjof Capra's Tao of Physics covers similar territory.

Over the years, I find the optimistic belief in Capra's books that people are constrained to work in ways similar to natural ecosystems to be increasingly strained; national economies built on stock markets and dominated by both the results of elections and the effectiveness of propoganda will inevitably sacrifice long- over short-term considerations (and the needs of the many in favour of the privileged few), especially when a short term of 4 years is sufficient to enable a small upper-tier ecosystem of folks to win significant personal advantage at the expense of significant (and sometimes irreparable) long-term harm to all (no, I'm not opposed to capitalism; damned if I know what can really work, but I suspect that automating and enabling high levels of verifiable transparency into the inputs and actions of government and the justice system by a country's citizens might help to counter the fact that money can buy a better class of election or legal campaign with a near-direct influence on the outcome; that level of automation would constitute real "e-government", but who in government would ever fund let alone adopt such a thing... ;).

However I believe that regardless of our actions and choices, Capra's notion of sustainability (especially described in his Ecology and Community article) is applicable; also, it is certainly useful to people working at the finer granularity of companies and workgroups - I see evidence of some of his ideas in the success of open-source communities and the notion of business co-opetition which is gaining acceptance.

(2004-09-15 11:20:57.0) Permalink

Philip K. Dick, virtual humanity and (oops) Hellboy

Betimes, I'm a bit of a sci-fi buff, and looking at the blogs of some folks here (richb, Scott Hudson, James C. Liu and grahamm) on bsc I guess it often goes with the territory of being an engineer. It was kind of funny for me to see Isaac Asimov's "I, Robot" make it to the silver screen as the paperback with its shining silver face was the first book I ever chose for myself (I was 10 and I still remember thinking "wow, a robot that knows itself - would it have a soul, and if so, where would it come from?"; sadly Asimov never tackled those issues but he did create a lot of inventive stories by stretching the constraints of the 3 Rules of Robotics).

Recently I've indulged my reading habit with:

some fibre - a mixture of political analysis (Chomsky's Hegemony or Survival) with some pop-pol (Michael Moore's Stupid White Men), and light but not lightweight business (Tom Peters' colourful Re-imagine!)
plenty of brain-carb's (remember, they're good for you) and -candy (sweet stimulation) using a steady feed of classic but lateral SF, including:
- a first-time read of Philip K. Dick's awesome The Little Black Box (out of print, sadly as it contains many wonderful stories including We Can Remember It For You Wholesale (an astonishing story that was Hollywood-ized as Total Recall) and The Pre-Persons (against which my convenient rationales for certain choices simply dissolve); you can see some of PKD's "inventions" on technovelgy; I think the empathy box (and the black box from the title story of TLBB ) is an interesting way to think about how technology may create new ways to create social connections and to share or even simulate experiences; the Philip K. Dick: Reason, Mind, and Being article has some interesting analysis on PKD's stories about emphathy as the defining quality of humanity; (I actually wonder the opposite - maybe the virtual experiences that are possible even today with TV and movies can actually overload our empathic intelligence by providing us with the occasion for empathy without the opportunity to act); PKD has a heap of movie credits too if you like to stimulate your optic nerve as well as your brain
- a re-read of the first four in Douglas Adam's HHGTG series, just so that I could read the final Mostly Harmless in context
- a first read of William Gibson's Pattern Recognition and a re-read of Idoru - Gibson explores the same human territory as PKD, but does it in a relentlessly realistic modality using a unique technical and dense literary prose - if you haven't read Gibson, sample a few paragraphs of an online copy of Neuromancer or Virtual Light or Mona Lisa Overdrive

But just for a change, I've over-dosed on media this last week with a whole 5 hours of TV (I watched local news a few nights, and did double-damage with Kill Bill 1 and 2 on rental VHS - definitely fun-ky). And I saw Hellboy last night (a slight but pretty entertaining movie, entirely due to Ron Perlman's ability to invest his chunky red character with some attitude; best lines (honest): "crap", "crap", "crap" and towards the end he alternates to "damn"; it does a pretty good job of keeping some dark material ok for under 12's with the result that it's a bit lacking for adults). Speaking of comic-derived movies, I guess there's not much hope that the few comics (oops, graphic novels) I've sampled since I was 13 like Hellblazer and Sandman (forget the horribly unimaginative Harry Potter franchise - the Books of Magic are Neil Gaiman's real thing) will in any way survive the transition to moviedom. If only Peter Jackson was a closet Gaiman fan... guess we'll have to wait till he's done King Kong to see if he's going to stay in the SF/fantasy genre and maintain the quality of his LOTR films...

Read on, gentle reader...

(2004-09-13 11:32:47.0) Permalink Comments [1]

Java and Desktop/Platform Integration

Ok, we all know that Java has the goal of enabling application portability to the most important platforms and devices, which in turn enables the largest possible market or user base for the application. But every now and again, I wanted to write a really slick desktop app that needed to do something on a platform that I could not do using a J2SE API. Shock, horror!

Although JNI is fine as a low-level universal bridge to native API's, it limits portability to the platforms supported by that native API. For those cases, it would be nice to have a portable Java API (perhaps not part of J2SE) but which includes appropriate a native backend "driver" to each of the platforms I need to integrate with (a little like Java Media Framework and its platform performance packs).

Well, the good news is that there are excellent solutions for many use cases, and there are some standard techniques for writing your own portable abstract API for use in your Java applications. I wish there was a good site to locate them all, but I don't know one - so I'd like to share a few of the best ones that I know about with you.

JDIC

The JDIC (Java Desktop Integration Components) project provides a set of independent API's under a common license (LGPL) and allow a portable Java application to use native features on many desktop platforms; your 100% pure Java code can:

Launch or even window-embed a native browser (such as IE or Mozilla)
Use the mailto: URL syntax to launch the desktop's registered mail application, open a Compose Mail dialog and set some attributes (subject, to, cc, etc.)
Launch the desktop's registered handler application for a file type (such as a document or media)
Register an application (such as your Java app) as the handler for a file type

The JDIC Packager is incredibly useful; if you create a Java Web Start (JNLP) setup to allow your application to be installed from any browser, you can use Packager to automatically create a native installer for Windows (MSI), Linux (RPM) and Solaris (PKG). Stunning stuff!

There are other JDIC projects in the incubator phase, including a Screensaver SDK (wrapper to allow a 100% pure Java screensaver to be integrated into the native screensaver system on Windows or X11 (Linux/Solaris) systems and the Tray Icon API (which appears in some respects to be even better than Systray for Java).

Java Service Wrapper

JDIC is far and away not the only resource for developers who need desktop integration. There is also the very useful Java Service Wrapper which can enable a Java application to:

Run as a Windows service or as a UNIX daemon
Re-start automatically if it terminates or hangs
Have platform-dependent JRE configuration properties specified in a plaform-independent manner (using a configuration file), eliminating the need to write a platform-specific or path-dependent launch script
Have its console output re-directed to a file or a system logger (Windows events or Linux/Solaris syslog)

While JDIC provides API's that allow you to use features of the platform, Java Service Wrapper enhances your existing application to make it run better and more reliably on multiple platforms - very nice, to put it mildly.

OpenOffice.org technologies

Finally there are the OpenOffice.org technologies - UNO and the OfficeBean.

UNO (the ambitious acronym stands for Universal Network Objects) is a middleware for writing components or exposing API's that may be used from multiple languages, such as Java, C, C++, Python and even .NET-based languages. You can also implement components in most of these languages. The result is language- and platform-neutrality; component programmers can be productive in their favourite language, while application programmers can use those components in their preferred language. Whoohoo!

And the best part about the OfficeBean is that it is a component API written using UNO that allows an application to use or window-embed a pre-installed OpenOffice.org-based application, such as OpenOffice.org itself or a commercial product like StarOffice. UNO was designed to make it easy to add support for other languages simply by implementing a UNO Bridge.

Down to cases

Ok, so that's my shopping list of Java/desktop integration technologies - let's look at some use cases for these technologies; these are just a few that occur to me, using the example of writing a mail client using Java and Swing:

you want to launch mail attachments using the standard registered desktop application (which could be native or Java) - simply save the attachment to a temporary file and use the JDIC-Desktop API to launch the app - job done
you would like to give your users the option to use a full-power office productivity application to compose mail messages - you just need to write a simple HTMLEditorWrapper interface that can use either Swing's HTMLEditorKit or the OfficeBean - no problemo
you want to render HTML mail messages quickly, but you also want to give the user full support for web standards if they need it; similar to the HTMLEditorWrapper you can create a HTMLViewerWrapper which can use the lightweight Swing HTMLEditorKit to display HTML, or you can use the JDIC-Browser API to embed the native browser in your Swing UI - quick and easy
you want to be able to view document attachments right inside of your Java mail client; you can use the OfficeBean and OpenOffice.org-based applications to display Microsoft Office or OASIS Open Office (standard OpenOffice.org) documents - powerful

Roll your own Java/desktop integration API

I mentioned Java Media Framework earlier as an example of how you can provide a portable API that can use native components, but perhaps the best example of the right architecture is JDBC which provides an API for application programmers and an SPI (service provider interface or "plug-in" interface) that is used to develop installable drivers for different kinds of databases and/or platforms, including databases with native API's. Because each backend database (Oracle, Postgres, HSQLDB, etc.) is supported using an installable "driver", updates to the driver for a specific backend can be obtained or distributed independently of the wrapper/abstraction API and the applications that use it. Similarly, a Java/desktop integration API should consist of a portable abstract API, a mechanism for finding the installed set of drivers or plug-ins, and an SPI for writing and installing a driver. For example, the JDIC-Browser API could have one installable "driver" for Mozilla support, and another for Internet Explorer, etc; the best part is that the driver can be shipped with the backend (e.g. Mozilla) so that it is up-to-date, and because the SPI is Java, the driver can be linked reliably.

Epilogue

There are certainly more Java/desktop integration API's than I mentioned above. I'd like to leave you with some technologies and techniques for implementing your own portable Java abstract API.

JOGL - an OpenGL wrapper technology used for example in Sun's Looking Glass project; JOGL is an example of a Java API that works with a portable native API; as long as the native API is feature complete and sufficiently portable for your needs, why not benefit from the RAD advantages of Java as a 100% portable application programming language and platform? (of course, Java has Java3D (now open-source) which is 100% portable and offers a very powerful model, but if you already know OpenGL top to bottom, JOGL makes good sense)
com4j - a Microsoft COM API wrapper generator; alternatively you can use JACOB if you want to use only COM IDispatch interfaces and you prefer not to need generated code, and there are even more options listed on the Java/Win32 Integration resources page; com4j is not a portable abstract API, but it provides an effective means to write a Windows-based "driver" for a portable 100% pure Java abstract API
SWT - I don't favour this technology because it attempts to solve a problem that's already solved in a 100% portable way by J2SE Swing; however, SWT's implementation approach (exposing the native windowing API via JNI using a 1-1 mapping to Java native methods) is worth considering as it has the advantage that most of the native adaptation code is in Java, although it has the disadvantage that it requires many fine-grained calls from Java to native code which impacts on performance

Hmm, that was a little more than I planned for one blog, I hope some of it was interesting to you.

(2004-09-10 01:06:31.0) Permalink Comments [2]

More thoughts on Bookmarks

I noticed RichB's thoughtful Thoughts on Bookmarks blog - you don't need to wait for Google labs to solve this, just use Google search "collaborative bookmark" and you can see the wealth of work that's already been done in this space! I agree with Rich that one valuable feature of shared bookmarks is the fact that they are maintained by a group, but I think a collaborative bookmarking system can have some other equally useful attributes:

bookmarks can be checked regularly for update or availability by a network server
each bookmark may be linked at multiple nodes in a taxonomy, or even in multiple discrete taxonomies
you could subscribe to be notified to changes in specific taxonomies or categories
if the structure of the taxonomic hierarchy is a separate entity, it can easily be shared which has two benefits:
- users don't have to learn multiple taxonomic structures and the labels for folders in the hierarchy
- a user could create a federated view of multiple taxonomies by a subscription mechanism; this would allow you to subscribe to a shared bookmark taxonomy with folks who have similar interests or a compatible way of classifying bookmarks
could be capable of associating a title, description and keywords with a URL (the network effect of sharing a URL encourages the user to provide more detail)

A collaborative bookmark manager would complement the linking style of blogging nicely; blogs are great for providing topical links that have a short half-life, whereas a shared bookmarking tool would help to create more long-term value.

I think such a tool could have a natural fit with content management systems (CMS); instead of persisting transient web content to your own hard disk, you could request that a web-addressable page or document would be cached in your group's CMS (subject to legal limitations on storage due to content licensing). If you access a bookmark via the bookmark server, it could automatically redirect to the local cache.

Naturally there are issues of importing and synchronising bookmarks from existing end-user tools such as Mozilla or IE, but none of that is rocket science. It would also be desirable to be able to sync with public bookmark taxonomies like the DMOZ Open Directory; that could be helped if DMOZ exposed an RSS interface to complement it's HTML one.

Of course, RSS is a pretty effective way to expose the structure, content or updates of any collaborative bookmark taxonomy.

Follow-up: RichB comments that he was really thinking about a personal bookmark manager, but given his stated issues (1. the time it takes each person to setup this set of bookmarks that can help them with their day to day work. 2. knowing what that magic set of best bookmarks are.), I don't see an ideal solution for an individual. However if an individual's bookmark taxonomy had some structure (categories or folders) in common with a shared bookmark source, it would be possible to merge selected categories/folders of the shared bookmarks, either dynamically or a one-time static merge.

(2004-09-06 12:34:15.0) Permalink Comments [2]