Thursday Aug 05, 2004

Summit Strategies Effectiveness Rankings(Disclaimer: Before we get too deep into this, let me say that although most of our web at Sun is pretty good [Summit Strategies just ranked us as having the most usable and effective IT-related site for the 4th straight quarter... See graphic at right]... there are also a lot of bumpy experiences on our web at Sun. It's a constant battle to make common experiences seem graceful and well thought through. So, consider me an expert in what not to do as well as the reverse.)

In the last 10 years of working on the web at Sun, I've been involved with dozens of sites and hundreds of hours of usability testing on our web sites. A lesson I've taken away is that the root causes of bad or disjointed user experiences tend to boil down to a few key mistakes that companies repeat over and over.  Funny, friends at other major sites tell me they see the same kinds of mistakes, resulting in the same kinds of user experience problems.

Here's my prescription for getting things right and avoiding these perilous mistakes in your company:

1. Think 'online first' - Don't let folks in your company treat the web as an afterthought. Even if your business is not primarily transacted on the web, the web is probably an important front door for your customers to learn about your products, get support, and possibly take delivery of products and services.  If your products contain software or a service, they probably needs to call back to the headquarters site. Yet,  though it sounds ludicrous, many product teams in many companies spend all of their time on traditional media like printed brochures and advertising... and then try to repurpose stuff to the web as an afterthought. The result will always be a user experience ("UE") that falls flat at best and is highly confusing and frustrating for web users who by all rights ought to be the first to get previews of your new products or announcements and actually want an online relationship with you..

Martin Doing Walkthrough

Here's me showing walkthrough techniques to KRON-TV in San Francisco.

2. Do preliminary walkthroughs for each new online experience - I'm a big believer in previewing experiences so you know what you're going to get. So, we encourage groups that are launching new products or updating specific online experiences to do a dry-run of the experience early in the development process.  These early walkthroughs -- even if only 10% of your content is ready -- will give you a sense of how your visitors will experience the pages and interactive stuff you're putting on the screen.  Walkthroughs are easy:

  • Just wire up representative HTML templates in your favorite editor and leave notes for  where pictures etc will go. 
  • Or, if you're in early phases, draw out the different screens on scrap paper. (This crayon-on-paper approach is what we did when I ran java.sun.com a few years ago; brief glimpses in this video).
  • Then, as a small team, walk through what you want the experience to be, and make notes about missing steps, messages you want to get across on specific screens, cool ideas for demos, etc.
(BTW, we also do validating cognitive walkthroughs right before launch or even after something is live... more below.)

3. Please run usability tests - This is Web 101, but for God's sake if you're doing something new and different, bring in some customers or representative users early in the process and test out the concepts. And then, when you think you have a really great design, do it again. You'll find your 'great design' needs tweaking, and that the tweaks will make a big difference in the usability and utility of your site experiences. If you don't have a lab or usability experts on staff, engage a good vendor. Or post the designs on a secret spot on your site and walk through them with customers on the phone. At Sun, we usually run a couple of usability tests every month on some aspect of our web sites, and these tests have generated several big overhauls as well as hundreds of small improvements to our site experiences.

4. Beware of arrogant label-makers - Across the Internet, inexperienced marketing folks love to create oddball ("innovative") names for their products and web site functions. These end up in the links and navigational structures on your site, and frankly your users won't understand what the heck they mean. So they won't click. And you'll lose a customer or at least make one mad. At Sun, we try to use task-oriented or functional link labels and navigation, even when our marketing teams may have come up with highly creative names. We don't always succeed at this straightforward approach but we sure try hard. (Usability testing is one antidote to this problem -- put video clips of confused users on your intranet and play them for your marketing folks.)

5. Validate after launch - OK, you did an initial walkthrough as the design was developing, and (hopes are) some usability testing, and you got your marketing folks to allow you to change all the confusing link names and 30-word headlines. Your design should be perfect, right? Yeah, as if. The problem is, a thousand disconnects can happen between the time a design is final and the site actually rolls out.  So, in addition to pre-launch tests, we also often do cognitive walkthroughs and usability tests after a site has gone live, to validate that things are working as we expect. (There are 'big data' techniques for validating too, which I'll get into more in a future post.)

6. Get robust design frameworks in place - Another issue many companies have (including Sun, three years ago) is that lack of a comprehensive design framework that supports all of the sorts of tasks your customers are doing on the web.  Without this kind of framework -- which should include a master IA structure, visual approach, and components to support everything -- you'll end up reinventing and redesigning with each new project, and that will not only be expensive but will lead to disjointed navigation and confused customers. In case you're curious, some of the components for our main design framework are online at sun.com/smrc/web

7. Strive for 'usable' then 'beautiful' - Ideally, you want a web property that looks great and works efficiently. But don't fall into the trap of creating a visually rich environment that nobody can figure out how to navigate or on which they can't perform common tasks. It's easier to add beauty to a site that has a great IA (good navigational structure, clear labeling, etc), than the reverse. (Believe me, we have some bland index pages I'd like to spruce up... but knowing that customers find them usable in testing tells me that at least we have the right structure to start with.)

8. Use agencies if you don't have a big staff - There are some pretty good design agencies our there, and they've probably already done 10 or 50 great designs reminiscent of the problem you're trying to solve today. In the long run, it's often a lot more cost effective to have a good external agency do major design overhauls, rather than trying to squeeze them in as midnight projects with your existing staff. Some of the design agencies we've used at Sun include frog design, Dubberly Design Office, Quintus Design and SBI/Razorfish, but there are lots of them around.

9. Make sure you have good base technology - A surprising number of bad user experiences are created because of technology limitations. You can't show users different automated views of document or product indexes if you don't have a metadata taxonomy set up for your content and products; you can't run A/B tests if you're not running the technology to support them; you can't rotate ads automatically if you don't have an ad server. Et cetera.  Understand the experiences you need to create, and then get hip with the technology you need to support these designs.  But, don't let technology run the show. Start with your business objectives and the user experiences you want to create; then make sure your technology platform supports it. If your IT group says "don't worry, this technology will do everything', be wary. You don't want to straight-jacket your user experience because of some technology or architectural limitation.

10. Beat back siloed web sites - Oh, and I have this other giganormous nightmare and its one that most of my friends at other sites share: Siloed web sites that were launched for some specific business program and now live on in legacy, confounding user navigation and sucking up support resources. These little subsites are usually launched for laudable objectives, such as getting a specific business program underway, or creating a new service. Next thing, you look up and and given user experience (such as, say, getting product support) may cut across 3 or 4 different sites, each with its own slightly different navigational system and even potentially different looks.  If you have the power, just say 'no' to siloed web sites in your company, especially if the owners claim they need a unique navigational system or visual design frameworks that doesn't fit with anything else.

And finally...

11. Don't let your vice-presidents do web design - Senior management, at many companies, tends to offer design advice such as "we need a big red button on every page of the site promoting groundhog day" (well, or whatever). I hear horror stories from colleagues all the time about this. Their VPs are busy with a million things and usually they haven't had a whole lot of design training, but somehow find time to do button design. I'm sure you love your VPs, as we do ours; they're good at pointing out experiences that suck (which we then fix). And, they have great instincts and often good taste. But we still don't let them do direct web design.  They're helpful for telling 'what' you should focus on, but usually not great at the 'how' you should achieve it from a design or engineering perspective. So, when your VP doles out a 'how' to you, go back and gently ask for a 'what' instead. Ask them what sites they like, what companies they admire, what they're trying to accomplish with their button. You'll have a better design as a result, and probably hit a home run with your business objective to boot.

I guess there's a lot of unspoken background in the items above (walkthroughs, usability tests, A/B trials, design frameworks, etc). I'll try to cover these in some future postings.

This blog copyright 2008 by MartinHardee