pithy

Maze Day 2008

Wed, 30 Apr 2008 23:34:00 -0400

About once a year, K-12 kids with disabilities from all over North Carolina (and beyond) travel to UNC-Chapel Hill to take part in Maze Day. Throughout the day, the kids, their teachers, and their parents wander Sitterson Hall to try out the numerous games, applications, activities, and demos designed to help them learn and have fun at the same time.

The kids always have a blast, and this year was no exception. Some of the 21 projects demoed this year included the following:

Carolina…

http://mindtrove.info/maze-day-2008/

Blog moving

Fri, 25 Apr 2008 16:41:09 -0400

My defense is over (passed), it’s a Friday, and I now have time to do things like, say, finally setup WordPress on my own domain.

If you actually care to read everything I put on my blog (i.e., accessibility), update your reader. (Certainly the posts will be greater in quantity and higher quality now that my brain isn’t stuck in the thesis writing loop, right?) If you only care about my occasional GNOME-specific ramblings, fret not. I’m configuring my tumblr account to automatically import new WP posts tagged with “gnome” so they continue to show up here, and hence on Planet.

Validate your accessibility

Wed, 16 Jan 2008 19:46:00 -0500

Eitan committed a new plug-in for Accerciser that makes it dirt simple to find basic accessibility problems. You know, the ones that cause grief for apps like Orca, GOK, On-Board, etc. To use it, run Accerciser, point it at part of a GUI, click validate, and wait for the report.

The rules in the plug-in aren’t the greatest right now. But the plug-in is extensible with new rule sets called schemas. For instance, you could have a “Desktop” schema to check basic GUI problems, a “Web” schema to test document accessibility, and an “Orca” schema to check a program’s fitness for Orca scripting. The sky’s the limit, and I’m sure Eitan, Will, and company will come up with quite a few useful tests.

To ward off any fear brought on by the word “schema,” I should note that they’re really just Python modules with simple, three-method classes in them. For example:

class CheckFocusable(Validator):

  def condition(self, acc):

    # only test accessibles that have the action interface

    return acc.queryAction()

  def after(self, acc, state, view):

    # check an accessible after checking its descendants

    # acc is the accessible

    # state is a dictionary of whatever you need to store across tests

    # view logs errors, warnings, etc.

    pass

  def before(self, acc, state, view):

    # check an accessible before checking its descendants

    s = acc.getState()

    if not s.contains(STATE_FOCUSABLE):

      view.error('actionable widget is not focusable')

No more excuses for inaccessible apps now, right? :)

Spatial PulseAudio

Tue, 01 Jan 2008 13:50:56 -0500

In his interview about Pulse Audio in Fedora 8, Lennart Pottering mentions support for spatial sound as one of his future goals:

Spatial event sounds: click on a button on the left side of your screen, and the event sound comes out of your left speaker. Click on one on the right side of your screen, and the event sound comes of of the right speaker. It’s earcandy, but I think this could actually be quite useful, but only if we get better quality event sounds, than we have right now.

While spatialized event sounds may be “earcandy” as Lennart admits, there are other benefits of using 3D audio over mono sounds in certain applications. One interesting use concerns the separation of concurrent sound streams such that a user can distinguish and “pick out” one of many. The theory of auditory scene analysis (Bregman, 1990) says (among many other things) that we humans can better segregate different sound sources and select one for attentive processing if the acoustic and semantic properties of streams from distinct sources differ along certain dimensions while certain properties within a stream remain constant over time.

For instance, say I make two audio recordings of the same person speaking two different utterances. In one recording, the person says “What a lovely bunch of coconuts.” In the other, “That dog certainly has fleas.” If I mix these two recordings into a mono track with the two utterances starting at exactly the same time, you will have a hard time determining and understanding the two independent phrases. If I create a stereo sound, with one phrase played in the left speaker and another in the right, you’ll have a much easier time identifying the original phrases. (But you’ll likely have to listen to the sound more than once before you can repeat both phrases: another tenet of auditory scene analysis.) Better still, if I apply a head related transfer function (HRTF) to each recording such that the two utterances appear to come from the left and right side of you head in a 3D space, your task becomes even easier.

In other words, spatialized sounds aid segregation and selection of independent streams of speech and sound. In fact, research (McGookin, 2004) suggests that spatialization alone is sufficient to aid recognition of information encoded in properties of concurrent musical sounds (earcons).

Applying the concept of distinguishable sound streams to screen readers is an interesting endeavor. (Or, at least, I think so.) Screen readers currently rely on a single, serial stream of speech and sound to describe the multitasking, high-bandwidth graphical desktop. In a single stream design, reports of peripheral information outside the application focus are either non-existent, delayed, or interruptions, and can be easily missed. For instance, if a screen reader is busy reading an email when the user receives an instant message in another application, the screen reader has to decide whether to keep reading the email or announce the new message in some manner. If the screen reader interjects, the user might confuse the instant message content with that of the email or become annoyed with the interruption. If the screen reader decides to wait for the email reading to finish, the late announcement about the chat message runs the risk of being stale.

Worse yet, any single-stream announcement of the new message can be inadvertantly interrupted at any time by the next user command. In such a situation, unless the user tabs around looking for the new instant message or the chat program is set to play a sound every time a message is received (which still doesn’t indicate which of potentially many chats has the new message), the user may never learn of the existence of the new message.

Concurrent streams provide an answer to this peripheral awareness problem, but only if the screen reader can present them in a way that avoids masking other simultaneous streams. And this is exactly where the ability to spatialize sound helps. Without interrupting or modifying the stream of speech reading the email, another stream can pipe up and announce the new chat message with a sound, speech, or both according to the verbosity settings of the user (Zing! or “Message from Harvey” or “Harvey says ‘Hey! Stop reading your email and answer me! This is important!’”). As long as these streams are spatially separated according to some simple rules, the user will be able to effectively distinguish them, ignore one, listen to one, and switch attention back and forth between them.

Instant messaging is just one example of a modern desktop application that begs for concurrent streams in screen readers. Just from looking at my GNOME desktop I see a system monitor, the clock applet, my network status, a popup balloon, and a log monitor all updating in the background while I write this post. Of course, a user can’t cope with all of these event sources reporting at once. But that’s where the interesting design problems start: how do we construct a usable multi-stream auditory display?

An open source library supporting spatial sound is a fundamental building-block for this investigation (and I’m certain, others). I hope Lennart pursues it.

RTM + Gmail = task management goodness

Thu, 20 Dec 2007 07:38:47 -0500

RTM + Gmail = task management goodness

Android speech synth (where are you?)

Tue, 18 Dec 2007 13:22:46 -0500

I took a peek at the Google Android class hierarchy today. As far as UI goes, it looks like there’s great support for 2D/3D visuals. There’s some APIs for doing MIDI and sampled sound output. There’s even a class for doing speech reco.

What I don’t see is anything supporting synthesized speech output. That’s a bit depressing. It would be a huge boon to have an open environment for developing mobile audio apps. Talking cell phones can be a bit pricey because they’re primarily intended as assistive technologies (i.e., small market). But I can imagine a ton of applications with speech-displays that could be useful to sighted and blind users alike: listening to your email while you walk instead of reading it on a tiny screen, announcements about upcoming meetings in your calendar, voice-jockey-like naming of songs about to come up on your MP3 playlist, spoken caller ID, …

Perhaps it’s possible to add custom classes to support FreeTTS or some other Java-accessible engine. However, it would be much nicer to have the speech API in the platform itself so it’s available everywhere. Maybe they left it out because all the free engines are too resource hungry? Somehow, I can’t imagine something like espeak being too bulky for a mobile platform.

MS User Interface Automation coming to Linux

Thu, 08 Nov 2007 13:13:00 -0500

MS User Interface Automation coming to Linux:

“… Novell will develop and deliver an adapter that allows the UIA framework to work well with existing Linux accessibility projects and complement the investments made by IBM Corp. and others. Novell’s work will be open source and will make the UIA framework cross-platform while enabling UIA to interoperate with the Linux Accessibility Toolkit (ATK), which ships with SUSE Linux Enterprise, Red Hat Enterprise Linux and Ubuntu Linux. The UIA solution will ensure interoperability of nonvisual access to the next generation of software applications.”

If I’m reading the article right, there will be a bridge from ATK to UIA, but there is no mention of an adapter between UIA and AT-SPI. That means ATs on the Linux platform will have to deal with two disparate accessibility APIs, AT-SPI and UIA, depending on what an application implements.

Update

Steve Lee points out that the article can be taken to mean that the bridge will really be from UIA to ATK/AT-SPI, not the other way around. That’s much more logical. I admit confusion since UIA is the name of both the client and server-side API.

Accessible Minesweeper

Fri, 10 Aug 2007 22:29:50 -0400

Accessible Minesweeper:

I posted this article a few weeks ago, but never got around to advertising it. I wanted to think about using ARIA live regions in situations other than the ones commonly cited: tickers, chat logs, scoreboards, etc. Minesweeper is an interesting case because multiple elements can change simultaneously on the game board when the user reveals or flags a tile: multiple tiles flip, the game is won or lost, the timer ticks, the flag count decreases. How might an AT present all this information effectively?

"Mitt Romney — who recently faced questions about his common sense for strapping his dog in its..."

Mon, 30 Jul 2007 13:10:59 -0400

“Mitt Romney — who recently faced questions about his common sense for strapping his dog in its carrier to the top of his car during a 12-hour drive, causing the animal to defecate over his windshield — said the format is beneath his dignity.”

- Republicans Terrified of YouTube Debate

FC7 build

Wed, 11 Jul 2007 01:11:59 -0400

Pleasantly surprised to see LSR installed, built as an RPM, and ran without a hitch on FC7. The new default Festival voice is interesting.

Updated the LSR in retrospect document. I recently realized I never uploaded the final draft to the website.

Dapper

Sat, 07 Jul 2007 11:07:43 -0400

Problem: I want to track water conservation notices in my county, but ncwater.org does not offer feeds or alerts of any kind. It only has static HTML pages generated via form input.

Solution: My very own NC Water Management Dapp. I can now consume notices about water restrictions in an RSS feed, NetVibes module, XML doc, email alert, JSON, CSV, and about seven other formats.

I’d toyed with Dapper before. It sounded like an interesting concept: point Dapper to similar pages, show it what inputs generated that page, tell it what output is important on the page, and choose an output format for the scraped data. Admittedly, I shrugged it off. Why would I want to screen scrape a site? Doesn’t everyone provide RSS/Atom feeds, email alerts, text messages, etc. these days?

How easy I forget that not everyone is so tech-savvy. I can thank Google for making me so naive. “What do you mean I can’t get service X delivered in format Y to my device Z?!”

Anyways, now I see how truly useful Dapper can be. I spent about fifteen minutes teaching it:

What input to use on the Check water use restriction status page
What output from the county Water system summary page to scrape
How to group the output information
What output format to produce from the data

If you haven’t seen Dapper already, I recommend watching the demo or toying with it yourself. It appears capable of doing quite a bit more than what I tested today.

Lazy, hazy days

Wed, 04 Jul 2007 00:16:59 -0400

Three days off ahead. Time to cook some dogs and ‘burgs tomorrow, relax with Jackie and Zora. Then time to work on my thesis over the last two days of the week. I’ve spent entirely too much time hacking on mindtrove lately, but I’m happy with where it’s heading. I’m sure it has all sort of issues rendering in non-Firefox browsers since I haven’t had the willpower to enter the compatibility quagmire yet. But MochiKit sure is fun. :)

Now that the structure is decent, I think I’ll work on expanding the content, particularly the articles section. There are four on my wish list (+1 in my head) that I need to write down. It gets more important by the day as I start to forget all sorts of details about GNOME accessibility as PHP/JS/Dojo/Zend and Web accessibility start filing my head.

On the topic of forgetting, I just remembered that I still have some outstanding accessibility-related patches to commit for pyatk. They’re a first step toward making it possible for Python developers to implement accessible custom widgets (e.g., pycairo widgets). If you’ve got some free time and would like to hurry the work blocking meta bug #377642 along, please contact me.

Google Desktop Download

Thu, 28 Jun 2007 07:23:00 -0400

Google Desktop Download:

Now for Linux!

2007-06-17

Sun, 17 Jun 2007 21:01:44 -0400

Up early to mow the lawn and paint bookshelves for Jackie. Do shelves that hold bound paper count as a proper gift for a first anniversary? Four cases hold about 50% of our books. Dang.

Oh! The places you'll go!

Sun, 17 Jun 2007 21:00:20 -0400

For those who read this, I’m not longer working on GNOME accessibility projects for IBM. I’ve been transfered to the QEDWiki project, and may or may not have accessibility duties in the long run, but possibly some in the short term.

Nevertheless, I’m writing up some final documentation on LSR in hope that someone will find it useful:

A patterns document stating ways to solve different problems using the existing Perk scripting sytem.
A LSR in retrospect document starting what I would change, if I were starting over again today

I may work on making the latter changes myself in my spare time. I can see some usefulness is having a basic accessibility engine which I can use to script certain desktop actions or speech enable specific applications. For instance, maybe I want to listen to changes in a particular window (e.g. console with tail -f running, gaim chat window). Maybe I want some pre-recorded macros that perform a specific set of actions across applications that do not make their models public through dbus or another API. Or maybe I want a small toolbox window to augment an application with functions that aren’t immediately available in its toolbar.

I still plan on contributing to GNOME accessibility. I have some patches outstanding for pygtk that will allow widgets drawn with pycairo to become visible to AT-SPI clients (e.g. Orca, Accerciser, GOK). I just have to find time to work on them.

Home page

Tue, 12 Jun 2007 22:37:36 -0400

Home page:

Q: Why a static HTML site?

A: I need a place to collect tutorials and such. I hate using a blog for that purpose.

Blogopular

Fri, 08 Jun 2007 13:43:53 -0400

gb: cool. Now maybe my rank will go up. It's all about driving traffic to my blog ;-)
me: Yes. I know you're dying to become blogopular. You heard it here first: blogopular, regarded with favor by other blogger geeks.

Fixing Firefox Flash Foolishness

Fri, 08 Jun 2007 13:34:30 -0400

Fixing Firefox Flash Foolishness:

Gary has solutions for two accessibility problems with Flash applications:

Having to click on or mouse over a Flash embed before it receives other mouse and key events.
Having no access to right click events in embedded Flash.

Yes, this too shall pass

Fri, 01 Jun 2007 14:10:11 -0400

Yesterday, IBM decided to change strategies with respect to GNOME accessibility:

http://www-03.ibm.com/developerworks/blogs/page/schwer

Under this new plan, IBM is no longer supporting development of LSR, accerciser, pyatspi, AT-SPI::Collection, or Firefox/AT-SPI accessibility. These projects will not vanish, but the news does have an impact on each.

Accerciser

Eitan Isaacson is busy preparing Accerciser for inclusion in GNOME 2.20 under a grant from the Mozilla Foundation. Its development and documentation is far enough along that it should only require minimal future maintenance (i.e. bug fixes, updates to stay in sync with at-spi). As far as I know, Eitan plans to stay on as maintainer of Accerciser after the grant concludes. Even if Eitan does decide to leave, someone from the accessibility or automated testing communities could step up and take ownership over the code.

pyatspi

The Python bindings for AT-SPI are complete enough to power Accerciser today. Dogtail and LDTP are busy adopting them as well. Orca, as I understand it, plans to adopt them sometime in the GNOME 2.20 or 2.22 time frame. I will do my best to support pyatspi as problems arise until another member of the GNOME community is expert enough to own the binding. Since pyatspi is in the at-spi module, Li Yuan remains as the proper maintainer.

LSR

Development on LSR as a screen reader for GNOME will cease. We will make one last release of LSR 0.5.3 on Monday in line with GNOME 2.19.3. After that, the project will go dormant until various groups decide whether the LSR core will be used to drive other open source AT projects or if it will be abandoned altogether. Over the next few weeks, I will be updating and writing documentation in case the LSR core goes on to live in other projects. I will also reorganize the LSR wiki to de-emphasize the screen reader user content, and put the focus more on developer documentation. Contact me directly if you wish to discuss LSR.

Firefox and the Mozilla platform

Aaron Leventhal will remain the maintainer of accessibility for the Mozilla core. His priorities will now be: 1) ARIA support — Windows and Linux , 2) Firefox 3 accessibility regressions, 3) IAccessible2 and cross-platform issues.

Aaron will not be focusing on Linux accessibility support in the Firefox 3 timeframe unless it affects all platforms, API harmonization or ARIA support. The ATK/AT-SPI-specific support of XUL and HTML must now be via the existing community and module peers. Aaron will continue to be available to review bug fixes in those areas. Contact Aaron directly for details.

AT-SPI Collection
Ariel Rios has (or will) post his outstanding work on implementing the AT-SPI Collection interface. I’m not sure of his immediate plans, but he has expressed an interest in completing the work on his personal time. Contact Ariel directly for details. Documentation about Collection can be found at http://live.gnome.org/GAP/Collection.

—-

Personally, I am still extremely interested in accessibility and the GNOME desktop. The IBM decision means that I will no longer contribute to GNOME through my daily work, but I certainly plan to make contributions to GNOME as a hobbyist. In addition, I will gladly help anyone who wishes to develop or reuse any of the projects I worked on for the past two years. Feel free to contact me about them at any time.

Finally, I want to wish all the other accessibility developers on GNOME the best of luck. Keep fighting the good fight of making free software accessible to all those who want it.

2007-05-30

Wed, 30 May 2007 22:15:00 -0400

Some prep work for the Monday release of LSR 0.5.3. Crunched some outstanding bugs with the event dispatch and method chaining optimizations I’ve been doing in the core. Everything seems to be running smoothly now…

But this too shall pass.

Spent the rest of the day in a haze of phone calls, emails, and IMs. More on this frenzy soon enough.