Dan Baigent's Weblog: The ravings of a lunatic

Nothing has generated more interest in Sun's desktop initiatives than the Looking Glass ("LG") demo that Jonathan Schwartz and other Sun executives have been showing off at every opportunity. As one of the core contributors to the Looking Glass project and a co-inventor on some of its patents, I'm often asked about the purpose of Looking Glass. Is this really designed to make computers more useful? Or is it all about fancy "eye candy" and having something to talk about at trade shows?

At least part of the answer is that LG is very much about "eye candy" - and that's not a bad thing. LG has ignited a renaissance within Sun for interesting, engaging, high quality user interfaces. But for your average CIO, what would be the benefit of installing something like LG aside from interesting eye candy?

Keep in mind that LG is just a demonstration - a "thought experiment" as Jonathan likes to say. LG looks at how commodity hardware (specifically, hardware accelerated graphics) and commodity software (Linux, GNOME, OpenOffice, etc.) can be leveraged to create a compelling user interface and improve usability and productivity for desktop users. The assumption behind LG is that trends towards commodization of hardware will continue and that over the next 2 or 3 years you will not be able to get a PC that does not have some form of high-end graphics card (along with terabytes of disk space, gigabytes of memory and multiple gigahertz of clock cycles). The question LG tries to answer is if and how to these capabilities could be leveraged to create a more engaging UI and improve usability.

For example, it is not uncommon for a user to suffer from "frame overload" in 2D environments where there are too many windows open simultaneously. The user can no longer determine which one contains the work he is most interested in, especially when he can only view the top most window and perhaps a portion of the window title in the taskbar. LG uses transperancy on an unfocused window as a way to allow the user to see more of what's currently running on the desktop at any given time. Also, the ability to rotate all active windows in 3D space and view the "live" content as the frames rotate can provide a vehicle for quickly "picking" the window the user cares about. This is often presented in the LG demo as "eye candy" (rotating several running windows in 3D space), but could easily be implemented as a usability feature - a sort of "window grabber".

It is also an assumption of the LG project team that despite 3D capabilities, users will continue to interface with a computer primarily via 2D devices like a monitor, a keyboard and a mouse or other pointing device. Even though things like the way Tom Cruise interfaced with his computer in the movie "Minority Report" are entertaining, the LG team does not assume that such an interface really adds anything to the usability of a computer. In fact, we believe it would just cause a lot of spilled coffee. And the LG team also believes that the majority of applications will be "legacy" 2D applications (browsers, office productivity suites, media players, etc) that will take some time, if ever, before they recreate themselves as 3D applications. LG imagines how rendering these 2D applications in a 3D space changes the context of the applications and provides avenues for usability improvements.

For example, another common usability problem in 2D desktops is the loss of root window context for modal dialog boxes. A modal dialog box is part of the user interface such as an error message, a preferences settings or a confirmation dialog that "pops up" over a running application ("parent" or "root" window). It must be dealt with (like clicking OK) before control can be returned to the application. Since a modal dialog box is a free floating frame like any other window, it is possible for the user to accidently move the modal dialog off of the root window and lose the context of which root window belongs to which modal dialog. The user is sometimes only aware that they can't access the root window, or that the root window is somehow not responding. By leveraging the new "real estate" in a 3D desktop and flipping the window over to display a modal dialog on the back of a window, the association of the dialog with the root window is never lost and it is more intuitive to the user that they must deal with the dialog before they can access the root window again. In the LG demo, this is sometimes shown as flipping over a window to write a note on a virtual "sticky pad", but the idea is that 2D windows in a 3D environment suddenly have a lot of new capabilities - some of which could enhance the overall user experience.

For those of us who have used Unix style desktops like CDE and GNOME, we are accustomed to the idea that a user can define multiple "workspaces" and switch between them as a way of handling multiple applications simultaneously (email/calendar in one workspace, web browser in another, spreadsheet in a third, etc.). Studies have shown that humans remember visual references (colors, images) more readily than names or numbers and that they remember spatial references between images (to the left, up, down, etc.) better than arbitrary references (blue, red, green, etc.). In the LG demo, we show how a set of image backgrounds that can be switched in a panoramic view (Mars, the Stanford campus, etc.) can help users manage multiple workspaces and leverage these mental reference techniques to keep track of what's running where.

And there is much more. Of course, none of these types of changes are themselves a reason to switch to a 3D environment like that demonstrated by project Looking Glass. However, combined with some nifty eye candy, these enhancements show how Sun views the possibilities of future of desktop user interfaces and helps explain why LG continues to generate interest and enthusiasm for our desktop solutions.