[DISCUSS] Camera plugin considerations in Android

[Richard Knoll <ri...@microsoft.com>]

Hey all,

Over the past week or so I have been conducting a review of the issues for the camera plugin in JIRA and have noticed two big areas of confusion that seem to plague users in Android. I want to start a DISCUSS thread because both of these involve larger decisions to be made.

1. Cropping Images (allowEdit)

There are a number of issues related to this posted in JIRA and it's one of the larger quirks on Android for the camera plugin. The problem is that we are using an undocumented and officially unsupported API for doing cropping and this is leading to unpredictable behavior. We have no control over what cropping Activity the user has on their phone and if those Activities even return their results in a way that is compatible with our plugin (e.g. the Gallery application). Unfortunately, there is no easy way to handle this issue as far as I know beyond implementing our own crop Activity or dropping in some library that does the same thing. Is that the avenue we want to pursue? I think it's a common enough problem to warrant the effort. At the very least I believe we need to have a much sterner warning in the docs about using this feature in Android.

2. Activity Destruction

This one is a documented issue where the OS can kill our Activity when it's pushed into the background if the device is on low memory. Unfortunately, this is an unavoidable fact of life in Android that creates a pretty big issue in plugins which call out to another Activity (this is not limited to the camera). There are two ways that I see to pursue a solution to this problem. The first is requiring plugins to handle it on a case by case basis (this pull request<https://github.com/apache/cordova-plugin-camera/pull/97> proposes such a solution). The issue with such an approach is that it encourages platform specific APIs for plugins, which I am against. The second approach would be to somehow persist the application's state across Activity creation/destruction. I'll admit that I am not familiar enough with the webview-native bridge to comment on that particular approach, but I am sure it's a non-trivial undertaking. It does have the obvious upside of being a general solution, however. I noticed in the pull request above that there was some mention of Jesse working on a solution in this direction. Is that still happening?

Thanks,

Richard

Re: [DISCUSS] Camera plugin considerations in Android

[Joe Bowser <bo...@gmail.com>]

Oops, Have to respond to my last comment:

On Fri, Oct 16, 2015 at 4:50 PM, Joe Bowser <bo...@gmail.com> wrote:

> Hey
>
> Comments are inline. Insights into weird behaviour ahead!
>
> On Fri, Oct 16, 2015 at 4:32 PM, Richard Knoll <ri...@microsoft.com>
> wrote:
>
>> Hey all,
>>
>> Over the past week or so I have been conducting a review of the issues
>> for the camera plugin in JIRA and have noticed two big areas of confusion
>> that seem to plague users in Android. I want to start a DISCUSS thread
>> because both of these involve larger decisions to be made.
>>
>> 1. Cropping Images (allowEdit)
>>
>> There are a number of issues related to this posted in JIRA and it's one
>> of the larger quirks on Android for the camera plugin. The problem is that
>> we are using an undocumented and officially unsupported API for doing
>> cropping and this is leading to unpredictable behavior. We have no control
>> over what cropping Activity the user has on their phone and if those
>> Activities even return their results in a way that is compatible with our
>> plugin (e.g. the Gallery application). Unfortunately, there is no easy way
>> to handle this issue as far as I know beyond implementing our own crop
>> Activity or dropping in some library that does the same thing. Is that the
>> avenue we want to pursue? I think it's a common enough problem to warrant
>> the effort. At the very least I believe we need to have a much sterner
>> warning in the docs about using this feature in Android.
>>
>
> Agreed.  This was done by a drive-by feature request and was only
> half-done properly, and I spent a bunch of time fixing it to get it to
> where it is now, where it mostly works on a Nexus device.  Right now we
> only support the AOSP/Google Photos cropping library and anything else that
> crops should probably implement that.  The problem is that this isn't a
> part of CTS, so we have no idea if every application that responds to the
> CROP intent actually works, or if it's broken.
>
> Now, there's definitely third party plugins that do cropping that people
> should use, and you can see a more reliable version of this in Untappd
> where it uses Aviary.  This will have to remain a third party plugin for
> very obvious reasons, but it's something that we can point users to.
> Alternately, if there's other photo editing libraries for Android that work
> as well, we can point them to those plugins as well.
>
>
>>
>> 2. Activity Destruction
>>
>> This one is a documented issue where the OS can kill our Activity when
>> it's pushed into the background if the device is on low memory.
>> Unfortunately, this is an unavoidable fact of life in Android that creates
>> a pretty big issue in plugins which call out to another Activity (this is
>> not limited to the camera). There are two ways that I see to pursue a
>> solution to this problem. The first is requiring plugins to handle it on a
>> case by case basis (this pull request<
>> https://github.com/apache/cordova-plugin-camera/pull/97> proposes such a
>> solution). The issue with such an approach is that it encourages platform
>> specific APIs for plugins, which I am against.
>
>
> I'm against that as well.  I also want an ICLA from the contributor before
> this gets added, assuming they haven't signed one already.
>

I'm only OK with this if we can't ever figure out the state thing, or the
state thing turns out to be impossible.



>
>
>> The second approach would be to somehow persist the application's state
>> across Activity creation/destruction. I'll admit that I am not familiar
>> enough with the webview-native bridge to comment on that particular
>> approach, but I am sure it's a non-trivial undertaking. It does have the
>> obvious upside of being a general solution, however. I noticed in the pull
>> request above that there was some mention of Jesse working on a solution in
>> this direction. Is that still happening?
>>
>
> This is the road to madness.  You can do this for a specific application
> using URIs, and I've done this years ago on a client project, but to do
> this for the general case would be extremely difficult.  You would have to
> load the plugin on startup instead of loading it when called, and you would
> have to immediately register the callback somehow.  The bigger issue is
> recovering the state of the application when the application is restarted,
> and this is an obvious drawback to Cordova as it exists today if you have a
> single page or multi page app that tries to create the illusion of multiple
> screens.  For example, a banking application that has a scan cheque method
> will wonder why the hell it's not back on the scan cheque flow and is
> instead on the login screen or the menu screen.
>
> It would be great if we can somehow pass the state to the Intent bundle
> such that when we get the intent result, we can get the extra that
> corresponds to the application state and process it, but trying to cram
> state into this is definitely non-trivial, and I gave up on this years
> ago.  That said, maybe there's some new APIs that can help with this.
>
>
>>
>> Thanks,
>>
>> Richard
>>
>
>

Re: [DISCUSS] Camera plugin considerations in Android

[Joe Bowser <bo...@gmail.com>]

Hey

Comments are inline. Insights into weird behaviour ahead!

On Fri, Oct 16, 2015 at 4:32 PM, Richard Knoll <ri...@microsoft.com>
wrote:

> Hey all,
>
> Over the past week or so I have been conducting a review of the issues for
> the camera plugin in JIRA and have noticed two big areas of confusion that
> seem to plague users in Android. I want to start a DISCUSS thread because
> both of these involve larger decisions to be made.
>
> 1. Cropping Images (allowEdit)
>
> There are a number of issues related to this posted in JIRA and it's one
> of the larger quirks on Android for the camera plugin. The problem is that
> we are using an undocumented and officially unsupported API for doing
> cropping and this is leading to unpredictable behavior. We have no control
> over what cropping Activity the user has on their phone and if those
> Activities even return their results in a way that is compatible with our
> plugin (e.g. the Gallery application). Unfortunately, there is no easy way
> to handle this issue as far as I know beyond implementing our own crop
> Activity or dropping in some library that does the same thing. Is that the
> avenue we want to pursue? I think it's a common enough problem to warrant
> the effort. At the very least I believe we need to have a much sterner
> warning in the docs about using this feature in Android.
>

Agreed.  This was done by a drive-by feature request and was only half-done
properly, and I spent a bunch of time fixing it to get it to where it is
now, where it mostly works on a Nexus device.  Right now we only support
the AOSP/Google Photos cropping library and anything else that crops should
probably implement that.  The problem is that this isn't a part of CTS, so
we have no idea if every application that responds to the CROP intent
actually works, or if it's broken.

Now, there's definitely third party plugins that do cropping that people
should use, and you can see a more reliable version of this in Untappd
where it uses Aviary.  This will have to remain a third party plugin for
very obvious reasons, but it's something that we can point users to.
Alternately, if there's other photo editing libraries for Android that work
as well, we can point them to those plugins as well.


>
> 2. Activity Destruction
>
> This one is a documented issue where the OS can kill our Activity when
> it's pushed into the background if the device is on low memory.
> Unfortunately, this is an unavoidable fact of life in Android that creates
> a pretty big issue in plugins which call out to another Activity (this is
> not limited to the camera). There are two ways that I see to pursue a
> solution to this problem. The first is requiring plugins to handle it on a
> case by case basis (this pull request<
> https://github.com/apache/cordova-plugin-camera/pull/97> proposes such a
> solution). The issue with such an approach is that it encourages platform
> specific APIs for plugins, which I am against.


I'm against that as well.  I also want an ICLA from the contributor before
this gets added, assuming they haven't signed one already.


> The second approach would be to somehow persist the application's state
> across Activity creation/destruction. I'll admit that I am not familiar
> enough with the webview-native bridge to comment on that particular
> approach, but I am sure it's a non-trivial undertaking. It does have the
> obvious upside of being a general solution, however. I noticed in the pull
> request above that there was some mention of Jesse working on a solution in
> this direction. Is that still happening?
>

This is the road to madness.  You can do this for a specific application
using URIs, and I've done this years ago on a client project, but to do
this for the general case would be extremely difficult.  You would have to
load the plugin on startup instead of loading it when called, and you would
have to immediately register the callback somehow.  The bigger issue is
recovering the state of the application when the application is restarted,
and this is an obvious drawback to Cordova as it exists today if you have a
single page or multi page app that tries to create the illusion of multiple
screens.  For example, a banking application that has a scan cheque method
will wonder why the hell it's not back on the scan cheque flow and is
instead on the login screen or the menu screen.

It would be great if we can somehow pass the state to the Intent bundle
such that when we get the intent result, we can get the extra that
corresponds to the application state and process it, but trying to cram
state into this is definitely non-trivial, and I gave up on this years
ago.  That said, maybe there's some new APIs that can help with this.


>
> Thanks,
>
> Richard
>