Willie Walker's Weblog

Flavio Percoco contacted me today to let me know about some recent work he has been doing. Well...it's way cool: he's managed to hack a software solution that uses the built in web cam on his system to control his mouse pointer. This is friggin' cool. We've all been talking about this for a long time, but talk is cheap. Flavio actually went ahead and did something. Way to go Flavio!

I've invited Flavio to submit a GOPA Task Proposal to flesh out the work further, and I have my fingers crossed that we'll get a good proposal with achievable and useful deliverables for this space. It is definitely something we've wanted for a while. Neat.

Just a quick note on some nice Linux.com articles by Bruce Byfield:

• GNOME focuses on accessibility -- with a little help from Mozilla and others, Bruce Byfield, February 27, 2008
• GNOME 2.22 focuses on utilities and standard applications, Bruce Byfield, March 17, 2008
• Orca accessibility app makes a whale of an impact, Bruce Byfield, April 07, 2008

Many thanks to Bruce for his work and attention to free open source accessibility!

In a past post, I talked about the investigation that Mark Doffman and Rob Taylor did to migrate AT-SPI to DBus. This work was generously funded by the Mozilla Foundation.

While multiple communities (e.g., GNOME, KDE, Open A11y, etc.) agree this was the right direction to head, accomplishing the task will be no small feat. It is not something even the most dedicated hacker can do in their spare time. Instead, it needs qualified professionals to do a bang up job. In my opinion, Mark and Rob from Codethink qualify for the position. It turns out that Nokia agrees, and has fully funded the effort. It is also great that Mike Gorse from Novell is involved -- like Mark and Rob, he's a good friend of the Orca project.

Woo hoo! Many thanks to Nokia for their courage to fund this work. Many thanks also to the Mozilla Foundation for their funding of the initial investigation. Many thanks also to Mark and Rob for persevering and digging up funding. What a way cool team effort across multiple communities. Open source accessibility rocks!

In a previous blog post, I mentioned work that we had been doing to get eSpeak working on Solaris. The process required several other packages in order for it to work, and none of these packages were really used by much of anything else (of interest to me, anyway ). So, I spent some time to write a driver for eSpeak that talks directly to the Solaris audio device. Gilles was very helpful in answering questions I had about the audio support design in eSpeak, and I managed to get something going pretty quickly thanks to his answers.

The solution I created isn't perfect, with the main limitation being that we can sometimes run into latency problems when canceling speech. The imperfection of my solution is that I use blocked write's to the audio device - the write won't return until the Solaris audio device's buffer has room to take all the data. These blocked writes can block a thread in eSpeak which prevents it from calling cancel until the write has finished. In theory, this seems horrible. In practice, it seems to occur only occasionally, and only under pretty heavy use. The reason this seems to work OK is that there's a thread in eSpeak's audio support that dishes things to the audio device on kind of an 'as-needed' basis, generally keeping in step with the capacity of the audio output device. It does this kind of 'accidentally', however, and it's not really meant to be an asynchronous audio buffer management system.

Instead of blocked writes, a better solution might be to create a whole separate thread for managing non-blocking communication with the audio device, but I wanted to see how far I could push things without introducing the extra complexity. The blocked-write solution seems to work fine, and we can always work on the more complex solution if we notice users complaining about cancel latency issues.

I then worked with the eSpeak folks to get this in for eSpeak 1.37. Yeah to Gilles and Jonathan for their help and support here. I then updated the SFEespeak.spec file to use the new eSpeak 1.37 release, eliminating dependencies upon a number of things. Cool beans. I'm now in the process of working with our internal project teams to phase FreeTTS out and eSpeak in. This is all to lay the groundwork for continued work on an accessible install for Project Indiana.

PS - based upon discussions with the people working on Solaris audio, using the direct Solaris audio support should continue to be forward compatible and should not experience/introduce audio device contention. This is good news and is much better than the audio mess we tend to see on other platforms.

Sure, building Firefox from sources is easy. Just choose the right compiler, put a good ~/.mozconfig file in place, check out the sources, and build it. Here's what I needed to do on my Solaris Express Developer Edition 01/08 box.

Get the right compiler. The compiler that comes with SXDE 01/08 will croak on the beloved jsinterp.c file as mentioned in the Solaris Build Prerequisites page. The problem is trying to figure out how to patch the compiler. It turns out that the Sun Studio compiler that comes with SXDE is not done via pkgadd, so you cannot really do much to patch it. Instead, you can just grab a Sun Studio Express build and untar it. Then, point your $PATH to the bin directory where you untarred it and set up some other environment variables as well. Here's some cut/paste from my ~/.bash_profile. I was lazy and didn't make a variable to hold the path to the place I unpacked the Sun Studio Express build. If I end up grabbing another build, I'll be more diligent and will probably just set up a symbolic link to point to my latest SUNWspro. Note also that I use JDS CBE, so that by the time this code snippet is used, PATH has already been modified to find all the JDS CBE goodies (e.g., like a 'make' that works):

export PATH=$HOME/StudioExpress/2008-02-25.ii/SUNWspro/bin:/usr/local/bin:$PATH:
/usr/openwin/bin:/usr/demo/jds/bin:/usr/sfw/bin:/usr/sbin:/opt/sfw/bin:/usr/bin/
X11
export CC=$HOME/StudioExpress/2008-02-25.ii/SUNWspro/bin/cc
export CXX=$HOME/StudioExpress/2008-02-25.ii/SUNWspro/bin/CC
export LDFLAGS="-L$HOME/StudioExpress/2008-02-25.ii/SUNWspro/lib -L/usr/sfw/lib -R$HOME/StudioExpress/2008-02-25.ii/SUNWspro/lib:/usr/sfw/lib"
export LD_LIBRARY_PATH=$HOME/StudioExpress/2008-02-25.ii/SUNWspro/lib

Create your ~/.mozconfig file. Yeah, easy, once someone else gives you the contents of their file:

ac_add_options --disable-tests
ac_add_options --disable-debug
ac_add_options --enable-optimize
ac_add_options --enable-application=browser
. $topsrcdir/browser/config/mozconfig 

Checkout the Firefox sources:

cvs -d :pserver:anonymous-AT-cvs-mirror.mozilla-DOT-org:/cvsroot co mozilla/client.mk
cvs -d :pserver:anonymous-AT-cvs-mirror.mozilla-DOT-org:/cvsroot co mozilla/browser/config/mozconfig
cd mozilla
make -f client.mk checkout

Build it. This worked fine on my box. Took about an hour:

make -f client.mk build > make.out 2>&1

Run it. Here's what I do:

~/mozilla/dist/bin/firefox

If you want to build Thunderbird, everything is pretty much the same, except you need to do a global substitution of 'browser' with 'mail' in your .mozconfig file and your cvs checkout.

That's it. I wrote this more for me than you, but I hope you find it helpful.

Last fall, I was headed out for a bike ride and saw this crazy young chick flying by with a contraption on the top of the car she was driving. I figured it was for Google Maps Street View, but I gave up looking for my picturesque chiseled body after a while.

Last week, I was doing various things and ended up on Google Maps, so I decided to try again. Well...looky there! I made Google Maps Street View. I'm famous. If you use Street View to search around the area, you'll see several different poses of me. Print 'em out. Hang 'em on your wall. Surround them with hearts and kiss them before you go to bed at night.

Last year, I wrote about a push that Rich and Joanie did to improve the magnification support provided by Orca. This work not only included exposing more of the functionality provided by the gnome-mag project (see an old review of gnome-mag), but also working with Carlos, the past gnome-mag maintainer, to improve the gnome-mag implementation.

Last week, at the California State University at Northridge's Center on Disabilities Conference (aka CSUN) we were able to have direct interaction with magnification users, getting their direct feedback on the magnification work that Joanie and Rich had done since last year's conference. When I was talking with users, I'd let them drive the machine, which was a Solaris Express Developer Edition 01/08 machine (GNOME 2.20) with GNOME 2.22 bits for Orca and the AT-SPI infrastructure. Every single end user I talked to was very positive about the support. Some of my favorite quotes were (and I paraphrase because I didn't write these down word for word):

• I just wasted a lot of money buying a commercial product. This has all I need. -- student
• You've obviously focused on the needs of users versus putting in the useless bells and whistles needed to demo this to people who don't need magnification. -- technology evaluator
• This is actually very usable, much better than the joke you were showing last year. -- end user

All these were very warming to me, even the "much better than the joke you were showing last year" comment. At last year's conference, I had absolutely no reservations stating that Orca's magnification support was woefully inadequate. It was truly not a usable solution - we all knew it and we didn't pretend or present it in any other light. But, it was something that could serve as a tool to solicit input from end users. Since last year, we obtained comments and direction from end users, and Joanie was also able to bring her years of training end users to the table. While we still have work to do to improve magnification, the work that Rich and Joanie has obviously made Orca's magnification support something that real users find compelling and useful. Way to go, Rich and Joanie!

Peter Korn forwarded me two YouTube videos talking about Orca use in India: "The first video is an eight and a half minute collection of three Indian news reports about a seminar that Mr. Krishnakant Mane held for the blind in India late last year, using Orca and Ubuntu Linux. The second video - titled 'True Vision' - is a 10 minute promotional piece by ELCOT (Electronics Corporation of Tamil Nadu Limited) in cooperation with the Government of Tamil Nadu, talking about their adoption and use of Ubuntu Linux and Orca to help the blind in India."

Hearing about this kind of stuff is pretty neat (thanks Peter!). As with the Linux Journal article, this is neat not because it's about a project I lead, but because of a much more important reason: a member of the accessibility community is emerging as a leader and a champion for open source accessibility in his country, and his country is listening. That just rocks! Many thanks to Krishnakant Mane for your energy and dedication. Your leadership will not only influence your country, but it may also influence the world.

Today, the GNOME Foundation Announced a Program to Sponsor Accessibility Projects. This program will help raise both awareness of accessibility among maintstream developers as well as improve the overall accessibility of the GNOME platform. With much trust from the organizations funding the effort, David Bolter, Luke Yelavich, and I are the judges for the program. I'm really excited about this!

Here's the message from GNOME Foundation Board Member, Behdad Esfahbod:

GNOME Foundation Announces Program to Sponsor Accessibility Projects
====================================================================

BOSTON, Mass—February 27, 2008 — The GNOME Foundation is running an
accessibility outreach program, offering USD$50,000 to be split among
individuals. This program will promote software accessibility awareness
among the GNOME community as well as harden and improve the overall
quality of the GNOME accessibility offering.

The program is sponsored by GNOME Foundation, Mozilla Foundation,
Google™'s Open Source Program Office, Canonical, and Novell. This is the
second in a series of outreach programs coordinated and run by the GNOME
Foundation.

“I'm excited about the GNOME accessibility outreach program because it
continues the promotion of compelling accessible design as part of the
mainstream developer culture. We believe the set of tangible and
achievable tasks outlined will help improve the already good
accessibility offering of the GNOME desktop,” said Willie Walker, Senior
Staff Engineer of Sun Microsystems, Inc.

GNOME Outreach Program: Accessibility starts accepting applications on
March 1st and will run towards the end of the year. There will be two
tracks to the program: In the first track accepted individuals will work
towards accomplishing one of the major projects nominated for the
program, earning US$6,000 and can take up to six months to complete the
task. The second track will reward contributors US$1,000 for fixing five
bugs out of a pool of accessibility bugs nominated by the program
judges.

Individuals interested in participating in the program should check out
www.gnome.org/projects/outreach/a11y. More information about the program
may be found at the same location.

Read the full announcement at:
http://www.gnome.org/press/releases/gop-a11y.html

The GNOME Accessibility Community has been busy the past couple months identifying tasks that fit within the constraints of the program. It's been a very positive effort, with representatives from many diverse groups all participating and providing excellent input and direction.

While the constraints of the program prevented us from doing some things (e.g., funding organizations rather than individuals, taking on really huge projects, etc.), I was pleased that we were able to address the issues of highest importance to GNOME Accessibility. We've also been able to break the tasks down into areas where developers and non-developers alike can participate.

Woo hoo!

PS - the task list is done for all intents and purposes. We just need to dot our i's and cross our t's. Look to the URL's above for more information.

Last year, George Kraft from IBM worked with the Linux Journal folks to help get an issue with a focus on accessibility. As a result of this, Eitan Isaacson and I wrote feature articles on Accerciser and Orca, respectively. They were accepted! Yeah!

My wife was bringing in the mail the other day and said "Why the heck did these people send us 3 copies of the Linux Journal? I'm calling them to make sure we're not being charged for three subscriptions." That was my cue to tell her to read the front cover. After a while, she got it: Orca and Accerciser were prominently displayed on the front. I then turned to page 70 and there it was: "Orca -- Take the Killer Whale for a Ride." On page 76 was Eitan's article: "Make Your Application Accessible with Accerciser." Neat.

The significance of this is not seeing my name in print. To me, the significance is that we have a widely read magazine bringing accessibility to mainstream thinkers and developers.

When I started working on accessibility 20 years ago, it was always a game of "Mother May I?" and continual negotiation with developers and management. Accessibility was always viewed as that once-in-a-while special interest segment you see on the nightly news -- sappy music playing in the background with the narrator using words such as "bravery," "overcoming hardships" and other content-free ilk meant to focus on the disability and tug at people's heartstrings. When push came to shove, though, we learned what people really thought.

Over the years, I've seen the mentality change. Yes, there are still those sappy special interest segments with sappy music narrated by some teary eyed tart, but we're also seeing people making accessible design part of their normal everyday thinking. People "get it." We see companies like Sun supporting leadership roles in creating and building accessible design into platforms such as GNOME. We see accessibility having an impact on the decisions being made to procure and deploy software. We see mainstream developers incorporate accessibility considerations into their applications. We see increased understanding that it is about independence, efficiency, and productivity for everyone.

These two articles represent another round of awareness being brought to the mainstream. They represent more exposure to more people, especially people who will "get it" and have the ability to make a difference. Many thanks to the Linux Journal for their conviction and courage in dedicating space to these articles.

Last month, I wrote about getting eSpeak working on Solaris Express and how that was one of the gating factors for me working on an accessible install for Project Indiana. Well, with eSpeak working, I began obsessing over what it would mean to get an accessible install working with Project Indiana. Luckily, I had a business trip coming up, and luckily that trip meant spending a lot of sleepless nights awake in a hotel room. OK, maybe I'm lying about being "lucky". My son cries every night that I'm gone (he really really cries and misses me dearly, as I do him -- it breaks my heart), but at least I had something to occupy me as opposed to pining away for my family the entire time.

So, the cool news is that I've made some great progress with a live CD based upon Project Indiana bits. It took a lot of work, and a bit of hacking, but I was able to use the distribution constructor to create my own CD that had Orca and eSpeak on it. Boot with the CD and Orca comes up talking. Run the installer, and Orca presents it to you. Really neat, and all the work only adds about 1Meg to the total size of the ISO image for the CD.

I'm now in the process of working with the Project Indiana team to fill in some of the missing pieces and smooth over the rougher edges. But, my hope is that we can keep the energy and excitement up to get something officially in place later this year. If we accomplish this, people with disabilities will be able to independently install Indiana in the privacy of their own homes without the need for assistance from others. That would totally rock.

PS - The folks involved with Project Indiana are awesome. They know their stuff and have been extremely helpful.

The eSpeak speech synthesizer is a very useful tool for providing free speech synthesis for a variety of languages. Gilles Casse did some great work in the past writing a gnome-speech driver for eSpeak, which led to a whole bunch of wonderful things for Orca. Unfortunately, some of the technology used didn't work so well on Solaris Express and I've been trying for quite some time to remedy the situation -- it's been over a year of on-and-off hacking (mostly off). Much of the problem is just the horrendous state that audio support is in across all platforms.

At last year's GUADEC, I asked around about emerging audio solutions and what work people might be doing in this space. One of the strong contenders is PulseAudio, and it happened that Lennart Poettering was there to give a talk. I attended the talk and was intrigued and began asking around about PulseAudio support for eSpeak. Later in the year, Gilles worked on PulseAudio support for eSpeak, and I finally spent the time yesterday and today putting things together on my Solaris Express Community Edition b79 x86 box. With a little hacking (see my notes), I was able to get it working with Orca on Solaris Express.

Big deal, you say? Well it is for me. This helps Open Solaris provide free accessibility support for people with visual impairments for a large number of locales. In addition, the very small footprint of eSpeak also helps get us one step closer to an accessible install for Solaris Express. I'm psyched about that. I've already played around with using Orca to access a demo version of Project: Caiman installer and have sent the development team feedback. If I get time, I will look more at the Slim Install work, especially the Distribution Constructor. It's probably an unrealistic dream, but I'd love to show off an accessible install at this year's CSUN. That would be totally cool.

As usual -- thanks everyone who helped! Open source and positive community collaboration puts a smile on my face.

When we first started working on Firefox accessibility via Orca, we were running into a number of bugs that basically forced us to do some very brute force navigation techniques. The core of this brute force navigation was to analyze the content character by character in a pretty rudimentary way. Following a common mantra of programming (1. Make it work, 2. Make it work right, 3. Make it faster/smaller), we didn't do any smart tricks to speed things up. Instead, we stayed on Step 2 for a while: we just wanted to get things working so we could learn more about the user experience. We also couldn't jump to Step 3 because we needed changes made to Firefox's Gecko layout engine.

After much work with the Mozilla team, we finally had a chance to work on Step 3 of the problem. Joanmarie 'I work for free because I love Orca so much' Diggs jumped into it with great fervor. Marco's recent blog entry describes the very positive end user impact of Joanie's work on the problem. While the perception of end users is king, Joanie did some further analysis to quantify the improvements, summarizing them in the Orca Performance Metabug:

              Revision 3387 (pre-Gecko performance) 
                               vs 
        Current Orca plus latest patch to Bug 506360 (rev.4)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                 1. Reading the Wiki by Line
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

         4166839 function calls in 201.149 CPU seconds
          958015 function calls in 139.656 CPU seconds

-----------------------------------------------------------------
                  getLineContentsAtOffset

        ncalls  tottime  percall  cumtime  percall
           372    0.956    0.003   50.363    0.135 
           148    0.007    0.000    4.011    0.027
-----------------------------------------------------------------
                       goPreviousLine

        ncalls  tottime  percall  cumtime  percall
            66    0.014    0.000   43.315    0.656 
            72    0.005    0.000   13.542    0.188
-----------------------------------------------------------------
                         goNextLine

        ncalls  tottime  percall  cumtime  percall
            86    0.014    0.000   39.971    0.465 
            72    0.005    0.000   12.087    0.168
-----------------------------------------------------------------
                  findPreviousCaretInOrder

        ncalls  tottime  percall  cumtime  percall
     6469/5001    0.314    0.000   12.537    0.003
       588/164    0.174    0.000    3.849    0.023
-----------------------------------------------------------------
                    findNextCaretInOrder

        ncalls  tottime  percall  cumtime  percall
    8688/26781    1.128    0.000   37.050    0.001 
       362/146    0.116    0.000    2.582    0.018 
-----------------------------------------------------------------
                        getExtents

        ncalls  tottime  percall  cumtime  percall
         32972   11.495    0.000   17.228    0.001
          1090    1.214    0.001    2.363    0.002
-----------------------------------------------------------------
                      updateBraille

        ncalls  tottime  percall  cumtime  percall
           154    0.111    0.001   22.425    0.146
           148    0.162    0.001    2.600    0.018

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     2. Tabbing through the Bugzilla Advanced Search Form
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

         3675201 function calls in 193.921 CPU seconds
          548400 function calls in  96.273 CPU seconds

-----------------------------------------------------------------
                         onFocus

        ncalls  tottime  percall  cumtime  percall
           115    0.006    0.000   75.807    0.659
           115    0.006    0.000   10.105    0.088

-----------------------------------------------------------------
                   locusofFocusChanged

        ncalls  tottime  percall  cumtime  percall
            58    0.005    0.000   75.714    1.305
            59    0.009    0.000    9.397    0.159
-----------------------------------------------------------------
                      guessTheLabel

        ncalls  tottime  percall  cumtime  percall
            35    0.007    0.000   25.423    0.726
            34    0.006    0.000    2.538    0.075

Way cool work, and many thanks to Joanie and the Mozilla team. This is a wonderful example of the open source community working together in a constructive way to solve very difficult problems. Yeah!

PS - If you're wondering how we got the numbers, please refer to the Orca Regression Testing page, or just know that we have been using the cProfile module for Python.

Andre Klapper posted 2007's GNOME Bugzilla Statistics. For me, it was great to see several members of the Orca team as well as the Orca project itself make the various lists.

The Orca team takes bugs very seriously and also uses Bugzilla to track nearly all of the Orca work. While it can be hard work, the hard work pays off. Congrats to the Orca team for all their work last year. 2008 is going to be just as good, if not better.

My inbox this morning contained a message regarding another year for the Blizzard Challenge, which is a competition to help with the understanding and comparison of research techniques in building corpus-based speech synthesizers. The challenge is an awesome idea in that it helps continue research and development in speech synthesis techniques -- something the world dearly needs. I also have sincere respect and appreciation for Alan Black, who is heavily involved in the challenge and is one of the world's leaders in speech synthesis research (he is *the* leader in my mind).

I participated as a test subject in the first challenge, and have been loosely monitoring the subsequent challenges. I remembered I didn't look at last year's challenge, so I took a look at it today. All neat stuff. Of particular interest to me was Marc Shroeder's paper on Mary TTS, which uses parts of FreeTTS. In a past life, I created/led FreeTTS, which we never really had the time to take as far as we had wanted. So, it was really great to see some of the work help with other people's research. Thanks Marc!