Android a11y, Whitelists and Offline Story

[Joe Bowser <bo...@gmail.com>]

Hey

The last week, I've been doing some preliminary work on Android
accessibility, and after fiddling with Cordova, I've come to some
pretty bad realizations:

1. Android Accessibility in a WebView is achieved the same way we do
PhoneGap, except that their JS is injected from a static server.  This
means that if the app is offline, it's not accessible to the blind

2. Most of Android isn't actually accessible, namely the Play Store,
which is how most people discover apps. "Play Store, Play Store, Play
Store, Play Store" doesn't seem super useful.

3. There's no real point trying to make accessibility work on
Gingerbread, since the accessibility libraries don't really work on
that platform, and aren't as robust.

4. Accessibility is HARD, especially when you're not blind.

5. The splashscreen says "Alert".  WTF?

So, one major issue that I ran into is that you have to whitelist
gstatic.com so that you can get the Accessibility Javascript. It'd be
great to see how the accessibility JS works and communicates with the
two interfaces that it adds to the WebView.

BTW: Here's the Java portion of Accessibility on the WebView:
https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/webkit/AccessibilityInjector.java

Any ideas on how we can take advantage of this, make it better when
we're offline. Right now, it just treats the webview as one giant
view.  Thoughts on this would be great.

Joe

Re: Android a11y, Whitelists and Offline Story

[Simon MacDonald <si...@gmail.com>]

More in-line comments:

> On Thu, Aug 29, 2013 at 1:25 PM, Joe Bowser <bo...@gmail.com> wrote:
>
>> 3. There's no real point trying to make accessibility work on
>> Gingerbread, since the accessibility libraries don't really work on
>> that platform, and aren't as robust.

Agreed, I've looked into this before and I haven't had any success
with anything less than Android 4.0.

>> 4. Accessibility is HARD, especially when you're not blind.
>
> Agreed. It takes an extra level of attention to detail to get that sort of
> stuff right, and you have to constantly stay on top of it.

Double agreed. We had to do a full release of adding a11y to our
product about 5 years ago and it was the most frustrating and yet
enlightening development experience ever.

>> 5. The splashscreen says "Alert".  WTF?
>
> Is this the Cordova app splashscreen? That's really strange, unless
> something is intercepting / reading out the prompt call from the bridge.
> Although in that case, I would expect it to happen pretty much continuously
> through an app.

The splash screen on Android is actually a full screen dialog. Even
though there is no title bar shown I'm betting it picks up the default
value "Alert". We should probably set it to something more accessible
like "Splash Screen". Although, then we get into internationalization
(i18n) issues for blind Dutch users :)

Simon

Re: Android a11y, Whitelists and Offline Story

[Ian Clelland <ic...@google.com>]

(I thought "Android a11y" was going to be Ally Ogilvie :) )

Some thoughts inline:

On Thu, Aug 29, 2013 at 1:25 PM, Joe Bowser <bo...@gmail.com> wrote:

> Hey
>
> The last week, I've been doing some preliminary work on Android
> accessibility, and after fiddling with Cordova, I've come to some
> pretty bad realizations:
>
> 1. Android Accessibility in a WebView is achieved the same way we do
> PhoneGap, except that their JS is injected from a static server.  This
> means that if the app is offline, it's not accessible to the blind
>

Do you mean that this is true for most android applications / web sites,
but that Cordova is currently better than that? (Or that PhoneGap is better
than that, at least?)

>
> 2. Most of Android isn't actually accessible, namely the Play Store,
> which is how most people discover apps. "Play Store, Play Store, Play
> Store, Play Store" doesn't seem super useful.
>
> 3. There's no real point trying to make accessibility work on
> Gingerbread, since the accessibility libraries don't really work on
> that platform, and aren't as robust.
>
> 4. Accessibility is HARD, especially when you're not blind.
>

Agreed. It takes an extra level of attention to detail to get that sort of
stuff right, and you have to constantly stay on top of it.

>
> 5. The splashscreen says "Alert".  WTF?
>

Is this the Cordova app splashscreen? That's really strange, unless
something is intercepting / reading out the prompt call from the bridge.
Although in that case, I would expect it to happen pretty much continuously
through an app.


>
> So, one major issue that I ran into is that you have to whitelist
> gstatic.com so that you can get the Accessibility Javascript. It'd be
> great to see how the accessibility JS works and communicates with the
> two interfaces that it adds to the WebView.
>

If there's an A11y plugin, then that can be added with <plugin.xml>. If
this should be a core feature, then we can easily build it in to the
whitelist, and enable it by default, like we do with file:///* and
android-asset://* urls.



Ian



> BTW: Here's the Java portion of Accessibility on the WebView:
>
> https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/webkit/AccessibilityInjector.java
>
> Any ideas on how we can take advantage of this, make it better when
> we're offline. Right now, it just treats the webview as one giant
> view.  Thoughts on this would be great.
>
> Joe
>

Re: Android a11y, Whitelists and Offline Story

[Andrew Grieve <ag...@chromium.org>]

Not something I've dealt with before :(

>From your description, perhaps we could write a plugin that bundles a copy
of the gstatic files and injects them when offline?


On Thu, Aug 29, 2013 at 1:25 PM, Joe Bowser <bo...@gmail.com> wrote:

> Hey
>
> The last week, I've been doing some preliminary work on Android
> accessibility, and after fiddling with Cordova, I've come to some
> pretty bad realizations:
>
> 1. Android Accessibility in a WebView is achieved the same way we do
> PhoneGap, except that their JS is injected from a static server.  This
> means that if the app is offline, it's not accessible to the blind
>
> 2. Most of Android isn't actually accessible, namely the Play Store,
> which is how most people discover apps. "Play Store, Play Store, Play
> Store, Play Store" doesn't seem super useful.
>
> 3. There's no real point trying to make accessibility work on
> Gingerbread, since the accessibility libraries don't really work on
> that platform, and aren't as robust.
>
> 4. Accessibility is HARD, especially when you're not blind.
>
> 5. The splashscreen says "Alert".  WTF?
>
> So, one major issue that I ran into is that you have to whitelist
> gstatic.com so that you can get the Accessibility Javascript. It'd be
> great to see how the accessibility JS works and communicates with the
> two interfaces that it adds to the WebView.
>
> BTW: Here's the Java portion of Accessibility on the WebView:
>
> https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/webkit/AccessibilityInjector.java
>
> Any ideas on how we can take advantage of this, make it better when
> we're offline. Right now, it just treats the webview as one giant
> view.  Thoughts on this would be great.
>
> Joe
>