You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cordova.apache.org by Aaron Charbonneau <am...@gmail.com> on 2013/03/12 18:58:35 UTC
Cordova Screen Capture Plugin for Android
Greetings! My name is Aaron Charbonneau, happy to be a new member of the
mailing list!
I have been developing a sceenshot plugin for Cordova to help facilitate
automation testing and debugging of Cordova apps, and I would love some
feedback on it. Currently Cordova provides a bunch of native functions
that allow you to do some cool stuff, but not much functionality to test
the apps that make use of them. Being able to take a capture of the screen
from within you app is a great way to automate testing or get additional
information for debugging an issue. Since there is no
Javascript mechanism for taking screen captures the solution would have to
be native, which fits nicely into the "gap" that Cordova/Phonegap bridges.
Any medium to large scale app can benefit greatly from automation testing
and any app can benefit from an extra debugging tool, and that is what I
hope this screenshot plugin can help achieve.
Currently the plugin offers 2 functions:
Capture():
* Take a capture of the current view, write that capture to a .png file
with the specified file name and sub directory of the sdcard (fallback to
emulated sdcard in the case there isn't an sdcard mounted)
* Able to create a sub-screenshot with a specified rectangle in order to
block out ui elements that may be variable, and also save space.
* Can take captures of images/dom elements (including canvas) that are
lager than the actual screen size
CaptureAndCompare():
* All the functionality of Capture()
* Perform a comparison between the captured image and a baseline image
located at the specified location in either the assets folder or the sdcard.
* User can specify per color channel tolerances as well as total pixel
tolerances to avoid false positives for the inevitable rendering
differences across devices.
* Optionally output a png file that contains the differences between the
actual and the baseline for debugging/triage purposes, two modes: binary
diff (all failing pixels appear as solid white) or the true differences
between pixels.
If you can spare some time, I would love it if you could take a look at the
api and parameters I have defined to make sure they adhere to Cordova
plugin best practices. The most crucial part would be in the plugin itself
at ScreenCapture.java:
https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
and captureAndCompare() are the two exposed functions)
I'm also interested to know you thoughts on it's usage. I started with the
mobile-spec testing framework and put some quick rendering tests into the
Autotest section as a viable use case. In order to get the WebView into
the state I wanted to capture, I had to implement a wait timer to allow the
view to update before taking the capture. Once the wait function
completes, the capture can be taken and everything from there is callback
based. I use the Jasmine waitsFor()/runs() blocks to make the tests run
synchronously. For some validation testing I made a compare function in
Javascript to run against the native compare. Turns out Java runs alot
faster than Javascript (surprise!) thus the native compare is about 2-5x
faster. All in all the process is fairly quick, the full captureAndCompare
with 3 file io's takes about 233ms on a Nexus One, and ~100ms on a Nexus 10
(256x256 image size).
Anyways here is the usage:
https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
And here's the JS wrapper for the plugin calls:
https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
Thanks for your time, and I look forward to ANY feedback positive or
negative.
Thanks,
Aaron
Re: Cordova Screen Capture Plugin for Android
Posted by Aaron Charbonneau <am...@gmail.com>.
"One more thing I noticed is that you're using mCaptureCount and mFileName
from different threads. You might run into errors if multiple calls are
happening at the same time. It'd be better to pass the fileName as a
function parameter, and maybe change the counter resetting logic to occur
only on the UI thread part."
-Ah yes good point, I will fix that.
"As for why it's choppier, is it because the throughput is now higher? If
you're not testing on a multi-core device, then the multi-threading might
have an adverse affect."
- Yes throughput is about 17% higher when spawning from the ui thread, but
what happens due to the jerky framerate is multiple captures happen on a
single render frame, then the animation skips a bunch of frames and our
next capture happens then, causing very inconsistent results. Spawning
from the core thread seems to yield much more consistent results where each
render frame is accompanied by a capture. This is on a multi-core Nexus
10.
-Aaron
On Wed, Mar 20, 2013 at 7:15 AM, Andrew Grieve <ag...@chromium.org> wrote:
> Glad you've found this useful. If you have any ideas for improving the
> docs, please feel free to make suggestions or file pull requests against
> the cordova-docs repo.
>
> One more thing I noticed is that you're using mCaptureCount and mFileName
> from different threads. You might run into errors if multiple calls are
> happening at the same time. It'd be better to pass the fileName as a
> function parameter, and maybe change the counter resetting logic to occur
> only on the UI thread part.
>
> As for why it's choppier, is it because the throughput is now higher? If
> you're not testing on a multi-core device, then the multi-threading might
> have an adverse affect.
>
>
> On Mon, Mar 18, 2013 at 7:08 PM, Aaron Charbonneau <amcharbo@gmail.com
> >wrote:
>
> > Ok thought this was all closed out, but I've hit a little snag with the
> > 'spawn background thread from ui thread'.
> >
> > For some reason for the throughput case, spawning the fileIO background
> > thread directly from the ui thread causes the ui to become much less
> fluid,
> > causing more jerking in a simple animation than spawning the fileIO
> thread
> > from the WebCore thread. The code is almost identical between the 2
> > methods, so it seems odd this would be happening. My only thought is
> that
> > the ui thread now has to hold on to some references of that background
> > thread? Perplexing, and not easy to find info on the interwebs about
> this.
> >
> >
> > Any thoughts?
> >
> > -Aaron
> >
> >
> > On Mon, Mar 18, 2013 at 1:57 PM, Aaron Charbonneau <amcharbo@gmail.com
> > >wrote:
> >
> > > Cool thanks for explaining that, alot of the information for Cordova
> > > plugin dev is a little out-dated, makes it hard sometimes to 'do the
> > right
> > > thing.'
> > >
> > > Ok I've made all the changes:
> > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
> > >
> > > I'll probably go forward with this approach once I've written some more
> > > tests to make sure there's no issues, and test out the additional
> > > functionality for throughput captures.
> > >
> > > Thanks again the help and feedback, was a very good learning
> experience.
> > > Please let me know if you see any other issues!
> > >
> > > Thanks,
> > > Aaron
> > >
> > >
> > > On Mon, Mar 18, 2013 at 1:44 PM, Andrew Grieve <agrieve@chromium.org
> > >wrote:
> > >
> > >> On Mon, Mar 18, 2013 at 4:38 PM, Aaron Charbonneau <
> amcharbo@gmail.com
> > >> >wrote:
> > >>
> > >> > "You could avoid the blocking queue if you dispatch the background
> > >> thread's
> > >> > runnable at the end of the UI thread's block."
> > >> >
> > >> > -Great idea, done.
> > >> >
> > >> >
> > >> > I traced through to see what sending a NO_RESULT PluginResult with
> > >> > keepCallback==true does, and it looks like a no-op
> > >> > (NativeToJsMessageQueue.java, addPluginResult()):
> > >> >
> > >> > boolean noResult = result.getStatus() ==
> > >> > PluginResult.Status.NO_RESULT.ordinal();
> > >> > boolean keepCallback = result.getKeepCallback();
> > >> > if (noResult && keepCallback) {
> > >> > return;
> > >> > }
> > >> >
> > >> > I'm sure I'm missing something here, is that really a no-op?
> > >>
> > >> Yep, that's a no-op and exists only to support the previous style of
> > >> Android plugins, where the execute method *had* to return a
> > PluginResult.
> > >>
> > >>
> > >>
> > >> > Did you
> > >> > mean for me to send a PluginResult.Status.OK with keepCallBack=true?
> > >> >
> > >> That's exactly what I meant :)
> > >>
> > >>
> > >> >
> > >> >
> > >> > Thanks,
> > >> > Aaron
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > On Sat, Mar 16, 2013 at 6:15 AM, Andrew Grieve <
> agrieve@chromium.org>
> > >> > wrote:
> > >> >
> > >> > > You could avoid the blocking queue if you dispatch the background
> > >> > thread's
> > >> > > runnable at the end of the UI thread's block.
> > >> > >
> > >> > > I see you send a success callback as soon as the image is
> captured,
> > >> but
> > >> > > before it is saved. This would improve throughput, but it might be
> > >> even
> > >> > > more useful if you set the keepCallback flag on it and then send
> > >> another
> > >> > > success response once you have the filename / comparison results.
> > It's
> > >> > the
> > >> > > filename that the app is probably interested in.
> > >> > >
> > >> > >
> > >> > > On Fri, Mar 15, 2013 at 9:41 PM, Aaron Charbonneau <
> > >> amcharbo@gmail.com
> > >> > > >wrote:
> > >> > >
> > >> > > > Ah thanks for clarifying that.
> > >> > > >
> > >> > > > I'm not sure how I can alleviate the ui thread any more than I
> am
> > >> > > > currently. It is already doing the bare minimum amount of work
> on
> > >> the
> > >> > > > there. Right now the only work done on the ui thread is
> > >> > > > view.capturePicture() which is the recommended thread for doing
> > that
> > >> > work
> > >> > > > (otherwise you get a warning):
> > >> > > >
> > >> > > >
> > >> > > > mUIThreadDone = false;
> > >> > > > cordova.getActivity().runOnUiThread(new Runnable() {
> > >> > > > public void run() {
> > >> > > > CordovaWebView uiThreadView = webView;
> > >> > > > picture = uiThreadView.capturePicture();
> > >> > > > mUIThreadDone = true;
> > >> > > > }
> > >> > > > });
> > >> > > > //rest of the work is done on core thread
> > >> > > > while(!mUIThreadDone) {}
> > >> > > > //write the picture to a file
> > >> > > > ...
> > >> > > >
> > >> > > > I could do the file io and compares on a different thread than
> the
> > >> > > > core one, but for the major use case, the core thread needs to
> > wait
> > >> no
> > >> > > > matter what to ensure that the file is written and the
> comparison
> > is
> > >> > > > complete, so I wouldn't expect much gain there.
> > >> > > >
> > >> > > >
> > >> > > > However, I've been experimenting with a more threaded version
> that
> > >> > > > does the file io and comparison on a background thread as you
> > >> > > > suggested. If you have time please check it out. While the code
> > is
> > >> > > > not as clean, I think it addresses the issues you raised Andrew.
> > >> For
> > >> > > > the major use case, performance is still the same as expected,
> but
> > >> if
> > >> > > > we want pure throughput on captures there's a big gain from
> about
> > 3
> > >> > > > captures per second (cps) to 10 cps on a Nexus 10.
> > >> > > >
> > >> > > >
> > >> > > > Here's the new plugin, with moar threading!
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
> > >> > > >
> > >> > > >
> > >> > > > Once again, thanks for all your feedback on this, I haven't had
> > much
> > >> > > > experience with threading in java before now.
> > >> > > >
> > >> > > >
> > >> > > > -Aaron
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Thu, Mar 14, 2013 at 8:10 AM, Andrew Grieve <
> > >> agrieve@chromium.org>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > My suggestion was just to prevent the UI thread from locking
> up
> > so
> > >> > that
> > >> > > > you
> > >> > > > > app doesn't appear to be frozen when the plugin is doing its
> > >> thing.
> > >> > > > >
> > >> > > > >
> > >> > > > > On Wed, Mar 13, 2013 at 5:03 PM, Aaron Charbonneau <
> > >> > amcharbo@gmail.com
> > >> > > > > >wrote:
> > >> > > > >
> > >> > > > > > Yah that's how I have it working now :) No problem with that
> > >> > approach
> > >> > > > for
> > >> > > > > > working with Jasmine.
> > >> > > > > >
> > >> > > > > > I made reference to running them asynchronously to respond
> to
> > >> > Andrew
> > >> > > > > about
> > >> > > > > > returning immediately after the bits are captured, which in
> > the
> > >> > case
> > >> > > of
> > >> > > > > > using that with Jasmine would only see a benefit IF we could
> > run
> > >> > > > multiple
> > >> > > > > > 'it' blocks at the same time.
> > >> > > > > >
> > >> > > > > > I think however Andrew was thinking of a different use case
> > >> where
> > >> > we
> > >> > > > > aren't
> > >> > > > > > limited by a framework that requires tests to run serially,
> > and
> > >> > > results
> > >> > > > > can
> > >> > > > > > just come back as they are ready, and be evaluated all
> > together
> > >> at
> > >> > > the
> > >> > > > > end,
> > >> > > > > > or not at all if you just want the captures and no
> evaluation.
> > >> > > > > >
> > >> > > > > > -Aaron
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Wed, Mar 13, 2013 at 1:53 PM, Braden Shepherdson <
> > >> > > > braden@chromium.org
> > >> > > > > > >wrote:
> > >> > > > > >
> > >> > > > > > > As far as I know, you're correct: there's no way to move
> on
> > >> the
> > >> > > next
> > >> > > > > it()
> > >> > > > > > > block before the first one is done.
> > >> > > > > > >
> > >> > > > > > > What's the problem with just making the tests wait? Why
> does
> > >> the
> > >> > > next
> > >> > > > > > it()
> > >> > > > > > > block need to start early? You can make your call to Java,
> > use
> > >> > > > waitFor
> > >> > > > > to
> > >> > > > > > > wait until the callback with the data is called, run your
> > >> > > > expectations,
> > >> > > > > > and
> > >> > > > > > > then move on to the next it() block. If the next it()
> block
> > is
> > >> > > using
> > >> > > > > the
> > >> > > > > > > result of this one, then they should be one it() block.
> > >> > > > > > >
> > >> > > > > > > Braden
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > On Wed, Mar 13, 2013 at 4:46 PM, Aaron Charbonneau <
> > >> > > > amcharbo@gmail.com
> > >> > > > > > > >wrote:
> > >> > > > > > >
> > >> > > > > > > > Hi Braden,
> > >> > > > > > > > Sorry maybe I am overlooking something. The way I
> > >> understand
> > >> > it
> > >> > > is
> > >> > > > > > that
> > >> > > > > > > a
> > >> > > > > > > > previous 'it' block needs to complete before the next
> one
> > >> can
> > >> > > run.
> > >> > > > > So
> > >> > > > > > I
> > >> > > > > > > > have various tests setup like this:
> > >> > > > > > > >
> > >> > > > > > > > it("renders something", function() {
> > >> > > > > > > > //draw something
> > >> > > > > > > > ....
> > >> > > > > > > > captureAndCompare(); //call the native function
> > >> > > > > > > > waitsFor(function() {
> > >> > > > > > > > return captureComplete; //set by the native function
> > >> callback
> > >> > > > > > > > }, "capture never completed", 10000);
> > >> > > > > > > > runs(function() {
> > >> > > > > > > > expect(captureResult).toBe(0); //expects difference to
> be
> > 0
> > >> > > > > > > > });
> > >> > > > > > > > });
> > >> > > > > > > > //next 'it' test
> > >> > > > > > > > ...
> > >> > > > > > > >
> > >> > > > > > > > The expect block needs to wait until there is a result
> > >> returned
> > >> > > > from
> > >> > > > > > the
> > >> > > > > > > > native captureAndCompare call. Therefore we use a
> > >> > waitsFor/runs
> > >> > > > > block
> > >> > > > > > > > there.
> > >> > > > > > > >
> > >> > > > > > > > But say we wanted to allow Javascript to run immediately
> > >> after
> > >> > > > > calling
> > >> > > > > > > that
> > >> > > > > > > > native function. The only thing you need to wait for is
> > >> that
> > >> > > Java
> > >> > > > > > > grabbed
> > >> > > > > > > > the bits, and the rest of the processing will be done
> 'at
> > >> some
> > >> > > > point
> > >> > > > > in
> > >> > > > > > > the
> > >> > > > > > > > future.' So as Andrew suggested you would return as
> soon
> > as
> > >> > the
> > >> > > > bits
> > >> > > > > > are
> > >> > > > > > > > captured, and allow JS to run again. The problem is
> that
> > >> our
> > >> > > > expect
> > >> > > > > > > block
> > >> > > > > > > > needs the results, so that has to wait, the only other
> > >> thing to
> > >> > > do
> > >> > > > > > would
> > >> > > > > > > be
> > >> > > > > > > > to run the next 'it' block.
> > >> > > > > > > >
> > >> > > > > > > > This is the part I didn't think was possible, can the
> next
> > >> 'it'
> > >> > > > block
> > >> > > > > > be
> > >> > > > > > > > run while the previous 'it' is still waiting to execute
> > the
> > >> > > > 'expect'
> > >> > > > > > > block?
> > >> > > > > > > > Or is there another approach all together that would
> > allow
> > >> > this?
> > >> > > > > > > >
> > >> > > > > > > > Thanks,
> > >> > > > > > > > Aaron
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <
> > >> > > > > > braden@chromium.org
> > >> > > > > > > > >wrote:
> > >> > > > > > > >
> > >> > > > > > > > > Jasmine does support asynchronous tests, and waiting
> an
> > >> > > arbitrary
> > >> > > > > > > period
> > >> > > > > > > > or
> > >> > > > > > > > > until some condition is true. Why does that not work
> for
> > >> > these
> > >> > > > > tests?
> > >> > > > > > > > >
> > >> > > > > > > > > Braden
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <
> > >> > > > > > amcharbo@gmail.com
> > >> > > > > > > > > >wrote:
> > >> > > > > > > > >
> > >> > > > > > > > > > Quick follow up question for the return values
> stuff:
> > >> > > > > > > > > > Does it make sense to have my functions called from
> > >> execute
> > >> > > > > > > (capture()
> > >> > > > > > > > > and
> > >> > > > > > > > > > captureAndCompare()) return an error string if one
> is
> > >> > > > encountered
> > >> > > > > > and
> > >> > > > > > > > > pass
> > >> > > > > > > > > > that back in callbackContext.error, then have
> execute
> > >> > return
> > >> > > > > true?
> > >> > > > > > > > > >
> > >> > > > > > > > > > Or is there a preferred way to get errors from sub
> > >> > functions
> > >> > > > back
> > >> > > > > > > into
> > >> > > > > > > > > > Javascript?
> > >> > > > > > > > > >
> > >> > > > > > > > > > Thanks!
> > >> > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > > > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau
> <
> > >> > > > > > > > amcharbo@gmail.com
> > >> > > > > > > > > > >wrote:
> > >> > > > > > > > > >
> > >> > > > > > > > > > > Thanks Andrew these are great suggestions!
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > About not needing the busy-wait in getScreenBits:
> > >> > > > > > > > > > > I'm not a fan of busy-waiting either.
> > >> > > > > > > > > > > Perhaps it's a side effect of Jasmine requiring
> > tests
> > >> to
> > >> > be
> > >> > > > run
> > >> > > > > > and
> > >> > > > > > > > > > > evaluated synchronously that I decided to use a
> > >> busy-wait
> > >> > > in
> > >> > > > > > > > > > getScreenBits.
> > >> > > > > > > > > > > If the Runnable calls
> CallbackContext.success/error
> > >> once
> > >> > > the
> > >> > > > > > bits
> > >> > > > > > > > are
> > >> > > > > > > > > > > captured (but not written to file) on the
> UIThread,
> > >> the
> > >> > > > > > Javascript
> > >> > > > > > > > side
> > >> > > > > > > > > > > will proceed to run before it has an image
> > url/compare
> > >> > > > result,
> > >> > > > > > most
> > >> > > > > > > > > > likely
> > >> > > > > > > > > > > causing that test to fail. As far as I know
> Jasmine
> > >> > > doesn't
> > >> > > > > > allow
> > >> > > > > > > > for
> > >> > > > > > > > > > > kicking off a bunch of tests and then gather all
> the
> > >> > > results
> > >> > > > at
> > >> > > > > > the
> > >> > > > > > > > end
> > >> > > > > > > > > > and
> > >> > > > > > > > > > > output pass/fails (each 'expect' function needs to
> > be
> > >> > > within
> > >> > > > an
> > >> > > > > > > 'it'
> > >> > > > > > > > > > block,
> > >> > > > > > > > > > > essentially that means one test must be evaluated
> > >> before
> > >> > > the
> > >> > > > > next
> > >> > > > > > > can
> > >> > > > > > > > > > > begin).
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > I can see the usefulness of allowing the capture,
> > file
> > >> > io,
> > >> > > > and
> > >> > > > > > > > > comparison
> > >> > > > > > > > > > > to be able to run asynchronously from the
> Javascript
> > >> > > though.
> > >> > > > In
> > >> > > > > > the
> > >> > > > > > > > > case
> > >> > > > > > > > > > > where you have your own test framework that CAN
> kick
> > >> off
> > >> > a
> > >> > > > > bunch
> > >> > > > > > of
> > >> > > > > > > > > tests
> > >> > > > > > > > > > > and just gather the results as they come in and
> do a
> > >> > batch
> > >> > > > > > > evaluation
> > >> > > > > > > > > at
> > >> > > > > > > > > > > the end, or in the case where you don't want to do
> > any
> > >> > > > > evaluation
> > >> > > > > > > in
> > >> > > > > > > > > your
> > >> > > > > > > > > > > app and just want to get captures as fast as you
> > can,
> > >> > > > > > asynchronous
> > >> > > > > > > > > could
> > >> > > > > > > > > > > increase performance there.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > I could try to implement it this way too, if you
> > think
> > >> > > > that's a
> > >> > > > > > > worth
> > >> > > > > > > > > > > while use case, maybe just provide an
> 'asynchronous'
> > >> flag
> > >> > > in
> > >> > > > > > > > > > CaptureOptions?
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > For doing the fileIO/compare using
> > >> cordova.getThreadPool,
> > >> > > is
> > >> > > > > > there
> > >> > > > > > > a
> > >> > > > > > > > > > > benefit to doing that if I still plan having the
> > >> > operation
> > >> > > > > block
> > >> > > > > > > the
> > >> > > > > > > > > > > Javascript? For asynchronous mode I see why that
> > >> makes
> > >> > > > sense.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > For the return values, I'll make those changes
> right
> > >> > away!
> > >> > > > > > Thanks
> > >> > > > > > > > > again
> > >> > > > > > > > > > > for providing your insight, hopefully I understood
> > it
> > >> > > > > correctly.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > -Aaron
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <
> > >> > > > > > > agrieve@chromium.org
> > >> > > > > > > > > > >wrote:
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >> Had a glance at your code. Things you may want to
> > >> > address:
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >> while(!mUIThreadDone) {}
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >> You shouldn't need to busy-wait. Pass the
> > >> > CallbackContext
> > >> > > to
> > >> > > > > the
> > >> > > > > > > > > > Runnable,
> > >> > > > > > > > > > >> and call success/error on it whenever you're
> done.
> > No
> > >> > need
> > >> > > > to
> > >> > > > > > have
> > >> > > > > > > > the
> > >> > > > > > > > > > web
> > >> > > > > > > > > > >> thread wait for it. Even better would be to use a
> > >> second
> > >> > > > > > Runnable
> > >> > > > > > > to
> > >> > > > > > > > > do
> > >> > > > > > > > > > >> the
> > >> > > > > > > > > > >> image compare / file IO on a background thread
> via
> > >> > > > > > > > > > cordova.getThreadPool()
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >> The return value to execute() should only be
> false
> > >> if an
> > >> > > > > invalid
> > >> > > > > > > > > command
> > >> > > > > > > > > > >> was passed. It's not meant to indicate the
> success
> > of
> > >> > the
> > >> > > > > > > operation.
> > >> > > > > > > > > You
> > >> > > > > > > > > > >> should be returning true for it.
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >> Instead of return "success"/"", it might simplify
> > >> > things a
> > >> > > > bit
> > >> > > > > > to
> > >> > > > > > > > > > return a
> > >> > > > > > > > > > >> boolean of true/false
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <
> > >> > > > > > > > lorin.beer.dev@gmail.com
> > >> > > > > > > > > > >> >wrote:
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >> > all the same, very impressive and faster then I
> > >> > > expected.
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >> > - Lorin
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron
> > Charbonneau
> > >> <
> > >> > > > > > > > > > amcharbo@gmail.com
> > >> > > > > > > > > > >> > >wrote:
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >> > > Thanks Lorin.
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> > > I did a quick test for throughput on an
> > animating
> > >> > > > canvas.
> > >> > > > > I
> > >> > > > > > > was
> > >> > > > > > > > > > able
> > >> > > > > > > > > > >> to
> > >> > > > > > > > > > >> > > grab 30 frames within a 5 seconds which
> equates
> > >> to
> > >> > > > around
> > >> > > > > > > 166ms
> > >> > > > > > > > > per
> > >> > > > > > > > > > >> > capture
> > >> > > > > > > > > > >> > > on a Galaxy Nexus. Unfortunately not fast
> > >> enough to
> > >> > > > > > automate
> > >> > > > > > > > > > >> something
> > >> > > > > > > > > > >> > > like video. As with any kind of capturing
> > >> system,
> > >> > > often
> > >> > > > > the
> > >> > > > > > > > > > overhead
> > >> > > > > > > > > > >> of
> > >> > > > > > > > > > >> > > copying bits around is enough to slow down
> the
> > >> > > operation
> > >> > > > > of
> > >> > > > > > > the
> > >> > > > > > > > > app.
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> > > -Aaron
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer
> <
> > >> > > > > > > > > > >> lorin.beer.dev@gmail.com
> > >> > > > > > > > > > >> > > >wrote:
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> > > > Hey Aaron,
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > > > very cool stuff, looking forward to
> checking
> > it
> > >> > out!
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > > > Question: any performance hit on the app by
> > >> using
> > >> > > > > Capture?
> > >> > > > > > > > > What's
> > >> > > > > > > > > > >> the
> > >> > > > > > > > > > >> > > > expected throughput on images taken in this
> > way
> > >> > > > (given a
> > >> > > > > > > > > > particular
> > >> > > > > > > > > > >> > > device,
> > >> > > > > > > > > > >> > > > say Galaxy Nexus)?
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > > > - Lorin
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron
> > >> > Charbonneau
> > >> > > <
> > >> > > > > > > > > > >> > amcharbo@gmail.com
> > >> > > > > > > > > > >> > > > >wrote:
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > > > > Glad you like it :)
> > >> > > > > > > > > > >> > > > > Yes in fact the actual capture makes use
> of
> > >> > > > > > > > > > View.capturePicture()
> > >> > > > > > > > > > >> > which
> > >> > > > > > > > > > >> > > > > actually grabs the entire document, then
> > that
> > >> > can
> > >> > > be
> > >> > > > > > > clipped
> > >> > > > > > > > > > down
> > >> > > > > > > > > > >> to
> > >> > > > > > > > > > >> > > the
> > >> > > > > > > > > > >> > > > > size/location of a specific element.
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > > > -Aaron
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal
> > >> Mocny <
> > >> > > > > > > > > > >> mmocny@chromium.org>
> > >> > > > > > > > > > >> > > > > wrote:
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > > > > Aaron,
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > > > I haven't even begun looking at your
> > >> > > > implementation,
> > >> > > > > > but
> > >> > > > > > > > I'm
> > >> > > > > > > > > > >> just
> > >> > > > > > > > > > >> > > going
> > >> > > > > > > > > > >> > > > > to
> > >> > > > > > > > > > >> > > > > > say it: this is awesome!
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > > > First question: When Capturing a DOM
> > >> element,
> > >> > > can
> > >> > > > > you
> > >> > > > > > > > > capture
> > >> > > > > > > > > > >> > 'body'
> > >> > > > > > > > > > >> > > to
> > >> > > > > > > > > > >> > > > > > grab it&children for a "full content
> > >> > > screenshot",
> > >> > > > or
> > >> > > > > > > does
> > >> > > > > > > > it
> > >> > > > > > > > > > >> have
> > >> > > > > > > > > > >> > to
> > >> > > > > > > > > > >> > > > be a
> > >> > > > > > > > > > >> > > > > > specific single element?
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > > > -Michal
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron
> > >> > > > Charbonneau <
> > >> > > > > > > > > > >> > > amcharbo@gmail.com
> > >> > > > > > > > > > >> > > > > > >wrote:
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > > > > Greetings! My name is Aaron
> > Charbonneau,
> > >> > > happy
> > >> > > > to
> > >> > > > > > be
> > >> > > > > > > a
> > >> > > > > > > > > new
> > >> > > > > > > > > > >> > member
> > >> > > > > > > > > > >> > > of
> > >> > > > > > > > > > >> > > > > the
> > >> > > > > > > > > > >> > > > > > > mailing list!
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > I have been developing a sceenshot
> > plugin
> > >> > for
> > >> > > > > > Cordova
> > >> > > > > > > to
> > >> > > > > > > > > > help
> > >> > > > > > > > > > >> > > > > facilitate
> > >> > > > > > > > > > >> > > > > > > automation testing and debugging of
> > >> Cordova
> > >> > > > apps,
> > >> > > > > > and
> > >> > > > > > > I
> > >> > > > > > > > > > would
> > >> > > > > > > > > > >> > love
> > >> > > > > > > > > > >> > > > some
> > >> > > > > > > > > > >> > > > > > > feedback on it. Currently Cordova
> > >> provides
> > >> > a
> > >> > > > > bunch
> > >> > > > > > of
> > >> > > > > > > > > > native
> > >> > > > > > > > > > >> > > > functions
> > >> > > > > > > > > > >> > > > > > > that allow you to do some cool stuff,
> > but
> > >> > not
> > >> > > > much
> > >> > > > > > > > > > >> functionality
> > >> > > > > > > > > > >> > to
> > >> > > > > > > > > > >> > > > > test
> > >> > > > > > > > > > >> > > > > > > the apps that make use of them.
> Being
> > >> able
> > >> > to
> > >> > > > > take
> > >> > > > > > a
> > >> > > > > > > > > > capture
> > >> > > > > > > > > > >> of
> > >> > > > > > > > > > >> > > the
> > >> > > > > > > > > > >> > > > > > screen
> > >> > > > > > > > > > >> > > > > > > from within you app is a great way to
> > >> > automate
> > >> > > > > > testing
> > >> > > > > > > > or
> > >> > > > > > > > > > get
> > >> > > > > > > > > > >> > > > > additional
> > >> > > > > > > > > > >> > > > > > > information for debugging an issue.
> > >> Since
> > >> > > there
> > >> > > > > is
> > >> > > > > > no
> > >> > > > > > > > > > >> > > > > > > Javascript mechanism for taking
> screen
> > >> > > captures
> > >> > > > > the
> > >> > > > > > > > > solution
> > >> > > > > > > > > > >> > would
> > >> > > > > > > > > > >> > > > have
> > >> > > > > > > > > > >> > > > > > to
> > >> > > > > > > > > > >> > > > > > > be native, which fits nicely into the
> > >> "gap"
> > >> > > that
> > >> > > > > > > > > > >> Cordova/Phonegap
> > >> > > > > > > > > > >> > > > > > bridges.
> > >> > > > > > > > > > >> > > > > > > Any medium to large scale app can
> > >> benefit
> > >> > > > greatly
> > >> > > > > > > from
> > >> > > > > > > > > > >> > automation
> > >> > > > > > > > > > >> > > > > > testing
> > >> > > > > > > > > > >> > > > > > > and any app can benefit from an extra
> > >> > > debugging
> > >> > > > > > tool,
> > >> > > > > > > > and
> > >> > > > > > > > > > >> that is
> > >> > > > > > > > > > >> > > > what
> > >> > > > > > > > > > >> > > > > I
> > >> > > > > > > > > > >> > > > > > > hope this screenshot plugin can help
> > >> > achieve.
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > Currently the plugin offers 2
> > functions:
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > Capture():
> > >> > > > > > > > > > >> > > > > > > * Take a capture of the current view,
> > >> write
> > >> > > that
> > >> > > > > > > capture
> > >> > > > > > > > > to
> > >> > > > > > > > > > a
> > >> > > > > > > > > > >> > .png
> > >> > > > > > > > > > >> > > > file
> > >> > > > > > > > > > >> > > > > > > with the specified file name and sub
> > >> > directory
> > >> > > > of
> > >> > > > > > the
> > >> > > > > > > > > sdcard
> > >> > > > > > > > > > >> > > > (fallback
> > >> > > > > > > > > > >> > > > > to
> > >> > > > > > > > > > >> > > > > > > emulated sdcard in the case there
> isn't
> > >> an
> > >> > > > sdcard
> > >> > > > > > > > mounted)
> > >> > > > > > > > > > >> > > > > > > * Able to create a sub-screenshot
> with
> > a
> > >> > > > specified
> > >> > > > > > > > > rectangle
> > >> > > > > > > > > > >> in
> > >> > > > > > > > > > >> > > order
> > >> > > > > > > > > > >> > > > > to
> > >> > > > > > > > > > >> > > > > > > block out ui elements that may be
> > >> variable,
> > >> > > and
> > >> > > > > also
> > >> > > > > > > > save
> > >> > > > > > > > > > >> space.
> > >> > > > > > > > > > >> > > > > > > * Can take captures of images/dom
> > >> elements
> > >> > > > > > (including
> > >> > > > > > > > > > canvas)
> > >> > > > > > > > > > >> > that
> > >> > > > > > > > > > >> > > > are
> > >> > > > > > > > > > >> > > > > > > lager than the actual screen size
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > CaptureAndCompare():
> > >> > > > > > > > > > >> > > > > > > * All the functionality of Capture()
> > >> > > > > > > > > > >> > > > > > > * Perform a comparison between the
> > >> captured
> > >> > > > image
> > >> > > > > > and
> > >> > > > > > > a
> > >> > > > > > > > > > >> baseline
> > >> > > > > > > > > > >> > > > image
> > >> > > > > > > > > > >> > > > > > > located at the specified location in
> > >> either
> > >> > > the
> > >> > > > > > assets
> > >> > > > > > > > > > folder
> > >> > > > > > > > > > >> or
> > >> > > > > > > > > > >> > > the
> > >> > > > > > > > > > >> > > > > > > sdcard.
> > >> > > > > > > > > > >> > > > > > > * User can specify per color channel
> > >> > > tolerances
> > >> > > > as
> > >> > > > > > > well
> > >> > > > > > > > as
> > >> > > > > > > > > > >> total
> > >> > > > > > > > > > >> > > > pixel
> > >> > > > > > > > > > >> > > > > > > tolerances to avoid false positives
> for
> > >> the
> > >> > > > > > inevitable
> > >> > > > > > > > > > >> rendering
> > >> > > > > > > > > > >> > > > > > > differences across devices.
> > >> > > > > > > > > > >> > > > > > > * Optionally output a png file that
> > >> contains
> > >> > > the
> > >> > > > > > > > > differences
> > >> > > > > > > > > > >> > > between
> > >> > > > > > > > > > >> > > > > the
> > >> > > > > > > > > > >> > > > > > > actual and the baseline for
> > >> debugging/triage
> > >> > > > > > purposes,
> > >> > > > > > > > two
> > >> > > > > > > > > > >> modes:
> > >> > > > > > > > > > >> > > > > binary
> > >> > > > > > > > > > >> > > > > > > diff (all failing pixels appear as
> > solid
> > >> > > white)
> > >> > > > or
> > >> > > > > > the
> > >> > > > > > > > > true
> > >> > > > > > > > > > >> > > > differences
> > >> > > > > > > > > > >> > > > > > > between pixels.
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > If you can spare some time, I would
> > love
> > >> it
> > >> > if
> > >> > > > you
> > >> > > > > > > could
> > >> > > > > > > > > > take
> > >> > > > > > > > > > >> a
> > >> > > > > > > > > > >> > > look
> > >> > > > > > > > > > >> > > > at
> > >> > > > > > > > > > >> > > > > > the
> > >> > > > > > > > > > >> > > > > > > api and parameters I have defined to
> > make
> > >> > sure
> > >> > > > > they
> > >> > > > > > > > adhere
> > >> > > > > > > > > > to
> > >> > > > > > > > > > >> > > Cordova
> > >> > > > > > > > > > >> > > > > > > plugin best practices. The most
> > crucial
> > >> > part
> > >> > > > > would
> > >> > > > > > be
> > >> > > > > > > > in
> > >> > > > > > > > > > the
> > >> > > > > > > > > > >> > > plugin
> > >> > > > > > > > > > >> > > > > > itself
> > >> > > > > > > > > > >> > > > > > > at ScreenCapture.java:
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >>
> > >> > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > >> > > > > > > > > > >> > > > > > > and captureAndCompare() are the two
> > >> exposed
> > >> > > > > > functions)
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > I'm also interested to know you
> > thoughts
> > >> on
> > >> > > it's
> > >> > > > > > > usage.
> > >> > > > > > > > I
> > >> > > > > > > > > > >> > started
> > >> > > > > > > > > > >> > > > with
> > >> > > > > > > > > > >> > > > > > the
> > >> > > > > > > > > > >> > > > > > > mobile-spec testing framework and put
> > >> some
> > >> > > quick
> > >> > > > > > > > rendering
> > >> > > > > > > > > > >> tests
> > >> > > > > > > > > > >> > > into
> > >> > > > > > > > > > >> > > > > the
> > >> > > > > > > > > > >> > > > > > > Autotest section as a viable use
> case.
> > >> In
> > >> > > order
> > >> > > > > to
> > >> > > > > > > get
> > >> > > > > > > > > the
> > >> > > > > > > > > > >> > WebView
> > >> > > > > > > > > > >> > > > > into
> > >> > > > > > > > > > >> > > > > > > the state I wanted to capture, I had
> to
> > >> > > > implement
> > >> > > > > a
> > >> > > > > > > wait
> > >> > > > > > > > > > >> timer to
> > >> > > > > > > > > > >> > > > allow
> > >> > > > > > > > > > >> > > > > > the
> > >> > > > > > > > > > >> > > > > > > view to update before taking the
> > capture.
> > >> > > Once
> > >> > > > > the
> > >> > > > > > > wait
> > >> > > > > > > > > > >> function
> > >> > > > > > > > > > >> > > > > > > completes, the capture can be taken
> and
> > >> > > > everything
> > >> > > > > > > from
> > >> > > > > > > > > > there
> > >> > > > > > > > > > >> is
> > >> > > > > > > > > > >> > > > > callback
> > >> > > > > > > > > > >> > > > > > > based. I use the Jasmine
> > >> waitsFor()/runs()
> > >> > > > blocks
> > >> > > > > > to
> > >> > > > > > > > make
> > >> > > > > > > > > > the
> > >> > > > > > > > > > >> > > tests
> > >> > > > > > > > > > >> > > > > run
> > >> > > > > > > > > > >> > > > > > > synchronously. For some validation
> > >> testing
> > >> > I
> > >> > > > > made a
> > >> > > > > > > > > compare
> > >> > > > > > > > > > >> > > function
> > >> > > > > > > > > > >> > > > > in
> > >> > > > > > > > > > >> > > > > > > Javascript to run against the native
> > >> > compare.
> > >> > > > > Turns
> > >> > > > > > > out
> > >> > > > > > > > > > Java
> > >> > > > > > > > > > >> > runs
> > >> > > > > > > > > > >> > > > alot
> > >> > > > > > > > > > >> > > > > > > faster than Javascript (surprise!)
> thus
> > >> the
> > >> > > > native
> > >> > > > > > > > compare
> > >> > > > > > > > > > is
> > >> > > > > > > > > > >> > about
> > >> > > > > > > > > > >> > > > > 2-5x
> > >> > > > > > > > > > >> > > > > > > faster. All in all the process is
> > fairly
> > >> > > quick,
> > >> > > > > the
> > >> > > > > > > > full
> > >> > > > > > > > > > >> > > > > > captureAndCompare
> > >> > > > > > > > > > >> > > > > > > with 3 file io's takes about 233ms
> on a
> > >> > Nexus
> > >> > > > One,
> > >> > > > > > and
> > >> > > > > > > > > > ~100ms
> > >> > > > > > > > > > >> on
> > >> > > > > > > > > > >> > a
> > >> > > > > > > > > > >> > > > > Nexus
> > >> > > > > > > > > > >> > > > > > 10
> > >> > > > > > > > > > >> > > > > > > (256x256 image size).
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > Anyways here is the usage:
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >>
> > >> > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > And here's the JS wrapper for the
> > plugin
> > >> > > calls:
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >>
> > >> > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > Thanks for your time, and I look
> > forward
> > >> to
> > >> > > ANY
> > >> > > > > > > feedback
> > >> > > > > > > > > > >> positive
> > >> > > > > > > > > > >> > > or
> > >> > > > > > > > > > >> > > > > > > negative.
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > Thanks,
> > >> > > > > > > > > > >> > > > > > > Aaron
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Andrew Grieve <ag...@chromium.org>.
Glad you've found this useful. If you have any ideas for improving the
docs, please feel free to make suggestions or file pull requests against
the cordova-docs repo.
One more thing I noticed is that you're using mCaptureCount and mFileName
from different threads. You might run into errors if multiple calls are
happening at the same time. It'd be better to pass the fileName as a
function parameter, and maybe change the counter resetting logic to occur
only on the UI thread part.
As for why it's choppier, is it because the throughput is now higher? If
you're not testing on a multi-core device, then the multi-threading might
have an adverse affect.
On Mon, Mar 18, 2013 at 7:08 PM, Aaron Charbonneau <am...@gmail.com>wrote:
> Ok thought this was all closed out, but I've hit a little snag with the
> 'spawn background thread from ui thread'.
>
> For some reason for the throughput case, spawning the fileIO background
> thread directly from the ui thread causes the ui to become much less fluid,
> causing more jerking in a simple animation than spawning the fileIO thread
> from the WebCore thread. The code is almost identical between the 2
> methods, so it seems odd this would be happening. My only thought is that
> the ui thread now has to hold on to some references of that background
> thread? Perplexing, and not easy to find info on the interwebs about this.
>
>
> Any thoughts?
>
> -Aaron
>
>
> On Mon, Mar 18, 2013 at 1:57 PM, Aaron Charbonneau <amcharbo@gmail.com
> >wrote:
>
> > Cool thanks for explaining that, alot of the information for Cordova
> > plugin dev is a little out-dated, makes it hard sometimes to 'do the
> right
> > thing.'
> >
> > Ok I've made all the changes:
> >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
> >
> > I'll probably go forward with this approach once I've written some more
> > tests to make sure there's no issues, and test out the additional
> > functionality for throughput captures.
> >
> > Thanks again the help and feedback, was a very good learning experience.
> > Please let me know if you see any other issues!
> >
> > Thanks,
> > Aaron
> >
> >
> > On Mon, Mar 18, 2013 at 1:44 PM, Andrew Grieve <agrieve@chromium.org
> >wrote:
> >
> >> On Mon, Mar 18, 2013 at 4:38 PM, Aaron Charbonneau <amcharbo@gmail.com
> >> >wrote:
> >>
> >> > "You could avoid the blocking queue if you dispatch the background
> >> thread's
> >> > runnable at the end of the UI thread's block."
> >> >
> >> > -Great idea, done.
> >> >
> >> >
> >> > I traced through to see what sending a NO_RESULT PluginResult with
> >> > keepCallback==true does, and it looks like a no-op
> >> > (NativeToJsMessageQueue.java, addPluginResult()):
> >> >
> >> > boolean noResult = result.getStatus() ==
> >> > PluginResult.Status.NO_RESULT.ordinal();
> >> > boolean keepCallback = result.getKeepCallback();
> >> > if (noResult && keepCallback) {
> >> > return;
> >> > }
> >> >
> >> > I'm sure I'm missing something here, is that really a no-op?
> >>
> >> Yep, that's a no-op and exists only to support the previous style of
> >> Android plugins, where the execute method *had* to return a
> PluginResult.
> >>
> >>
> >>
> >> > Did you
> >> > mean for me to send a PluginResult.Status.OK with keepCallBack=true?
> >> >
> >> That's exactly what I meant :)
> >>
> >>
> >> >
> >> >
> >> > Thanks,
> >> > Aaron
> >> >
> >> >
> >> >
> >> >
> >> > On Sat, Mar 16, 2013 at 6:15 AM, Andrew Grieve <ag...@chromium.org>
> >> > wrote:
> >> >
> >> > > You could avoid the blocking queue if you dispatch the background
> >> > thread's
> >> > > runnable at the end of the UI thread's block.
> >> > >
> >> > > I see you send a success callback as soon as the image is captured,
> >> but
> >> > > before it is saved. This would improve throughput, but it might be
> >> even
> >> > > more useful if you set the keepCallback flag on it and then send
> >> another
> >> > > success response once you have the filename / comparison results.
> It's
> >> > the
> >> > > filename that the app is probably interested in.
> >> > >
> >> > >
> >> > > On Fri, Mar 15, 2013 at 9:41 PM, Aaron Charbonneau <
> >> amcharbo@gmail.com
> >> > > >wrote:
> >> > >
> >> > > > Ah thanks for clarifying that.
> >> > > >
> >> > > > I'm not sure how I can alleviate the ui thread any more than I am
> >> > > > currently. It is already doing the bare minimum amount of work on
> >> the
> >> > > > there. Right now the only work done on the ui thread is
> >> > > > view.capturePicture() which is the recommended thread for doing
> that
> >> > work
> >> > > > (otherwise you get a warning):
> >> > > >
> >> > > >
> >> > > > mUIThreadDone = false;
> >> > > > cordova.getActivity().runOnUiThread(new Runnable() {
> >> > > > public void run() {
> >> > > > CordovaWebView uiThreadView = webView;
> >> > > > picture = uiThreadView.capturePicture();
> >> > > > mUIThreadDone = true;
> >> > > > }
> >> > > > });
> >> > > > //rest of the work is done on core thread
> >> > > > while(!mUIThreadDone) {}
> >> > > > //write the picture to a file
> >> > > > ...
> >> > > >
> >> > > > I could do the file io and compares on a different thread than the
> >> > > > core one, but for the major use case, the core thread needs to
> wait
> >> no
> >> > > > matter what to ensure that the file is written and the comparison
> is
> >> > > > complete, so I wouldn't expect much gain there.
> >> > > >
> >> > > >
> >> > > > However, I've been experimenting with a more threaded version that
> >> > > > does the file io and comparison on a background thread as you
> >> > > > suggested. If you have time please check it out. While the code
> is
> >> > > > not as clean, I think it addresses the issues you raised Andrew.
> >> For
> >> > > > the major use case, performance is still the same as expected, but
> >> if
> >> > > > we want pure throughput on captures there's a big gain from about
> 3
> >> > > > captures per second (cps) to 10 cps on a Nexus 10.
> >> > > >
> >> > > >
> >> > > > Here's the new plugin, with moar threading!
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
> >> > > >
> >> > > >
> >> > > > Once again, thanks for all your feedback on this, I haven't had
> much
> >> > > > experience with threading in java before now.
> >> > > >
> >> > > >
> >> > > > -Aaron
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Thu, Mar 14, 2013 at 8:10 AM, Andrew Grieve <
> >> agrieve@chromium.org>
> >> > > > wrote:
> >> > > >
> >> > > > > My suggestion was just to prevent the UI thread from locking up
> so
> >> > that
> >> > > > you
> >> > > > > app doesn't appear to be frozen when the plugin is doing its
> >> thing.
> >> > > > >
> >> > > > >
> >> > > > > On Wed, Mar 13, 2013 at 5:03 PM, Aaron Charbonneau <
> >> > amcharbo@gmail.com
> >> > > > > >wrote:
> >> > > > >
> >> > > > > > Yah that's how I have it working now :) No problem with that
> >> > approach
> >> > > > for
> >> > > > > > working with Jasmine.
> >> > > > > >
> >> > > > > > I made reference to running them asynchronously to respond to
> >> > Andrew
> >> > > > > about
> >> > > > > > returning immediately after the bits are captured, which in
> the
> >> > case
> >> > > of
> >> > > > > > using that with Jasmine would only see a benefit IF we could
> run
> >> > > > multiple
> >> > > > > > 'it' blocks at the same time.
> >> > > > > >
> >> > > > > > I think however Andrew was thinking of a different use case
> >> where
> >> > we
> >> > > > > aren't
> >> > > > > > limited by a framework that requires tests to run serially,
> and
> >> > > results
> >> > > > > can
> >> > > > > > just come back as they are ready, and be evaluated all
> together
> >> at
> >> > > the
> >> > > > > end,
> >> > > > > > or not at all if you just want the captures and no evaluation.
> >> > > > > >
> >> > > > > > -Aaron
> >> > > > > >
> >> > > > > >
> >> > > > > > On Wed, Mar 13, 2013 at 1:53 PM, Braden Shepherdson <
> >> > > > braden@chromium.org
> >> > > > > > >wrote:
> >> > > > > >
> >> > > > > > > As far as I know, you're correct: there's no way to move on
> >> the
> >> > > next
> >> > > > > it()
> >> > > > > > > block before the first one is done.
> >> > > > > > >
> >> > > > > > > What's the problem with just making the tests wait? Why does
> >> the
> >> > > next
> >> > > > > > it()
> >> > > > > > > block need to start early? You can make your call to Java,
> use
> >> > > > waitFor
> >> > > > > to
> >> > > > > > > wait until the callback with the data is called, run your
> >> > > > expectations,
> >> > > > > > and
> >> > > > > > > then move on to the next it() block. If the next it() block
> is
> >> > > using
> >> > > > > the
> >> > > > > > > result of this one, then they should be one it() block.
> >> > > > > > >
> >> > > > > > > Braden
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Wed, Mar 13, 2013 at 4:46 PM, Aaron Charbonneau <
> >> > > > amcharbo@gmail.com
> >> > > > > > > >wrote:
> >> > > > > > >
> >> > > > > > > > Hi Braden,
> >> > > > > > > > Sorry maybe I am overlooking something. The way I
> >> understand
> >> > it
> >> > > is
> >> > > > > > that
> >> > > > > > > a
> >> > > > > > > > previous 'it' block needs to complete before the next one
> >> can
> >> > > run.
> >> > > > > So
> >> > > > > > I
> >> > > > > > > > have various tests setup like this:
> >> > > > > > > >
> >> > > > > > > > it("renders something", function() {
> >> > > > > > > > //draw something
> >> > > > > > > > ....
> >> > > > > > > > captureAndCompare(); //call the native function
> >> > > > > > > > waitsFor(function() {
> >> > > > > > > > return captureComplete; //set by the native function
> >> callback
> >> > > > > > > > }, "capture never completed", 10000);
> >> > > > > > > > runs(function() {
> >> > > > > > > > expect(captureResult).toBe(0); //expects difference to be
> 0
> >> > > > > > > > });
> >> > > > > > > > });
> >> > > > > > > > //next 'it' test
> >> > > > > > > > ...
> >> > > > > > > >
> >> > > > > > > > The expect block needs to wait until there is a result
> >> returned
> >> > > > from
> >> > > > > > the
> >> > > > > > > > native captureAndCompare call. Therefore we use a
> >> > waitsFor/runs
> >> > > > > block
> >> > > > > > > > there.
> >> > > > > > > >
> >> > > > > > > > But say we wanted to allow Javascript to run immediately
> >> after
> >> > > > > calling
> >> > > > > > > that
> >> > > > > > > > native function. The only thing you need to wait for is
> >> that
> >> > > Java
> >> > > > > > > grabbed
> >> > > > > > > > the bits, and the rest of the processing will be done 'at
> >> some
> >> > > > point
> >> > > > > in
> >> > > > > > > the
> >> > > > > > > > future.' So as Andrew suggested you would return as soon
> as
> >> > the
> >> > > > bits
> >> > > > > > are
> >> > > > > > > > captured, and allow JS to run again. The problem is that
> >> our
> >> > > > expect
> >> > > > > > > block
> >> > > > > > > > needs the results, so that has to wait, the only other
> >> thing to
> >> > > do
> >> > > > > > would
> >> > > > > > > be
> >> > > > > > > > to run the next 'it' block.
> >> > > > > > > >
> >> > > > > > > > This is the part I didn't think was possible, can the next
> >> 'it'
> >> > > > block
> >> > > > > > be
> >> > > > > > > > run while the previous 'it' is still waiting to execute
> the
> >> > > > 'expect'
> >> > > > > > > block?
> >> > > > > > > > Or is there another approach all together that would
> allow
> >> > this?
> >> > > > > > > >
> >> > > > > > > > Thanks,
> >> > > > > > > > Aaron
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <
> >> > > > > > braden@chromium.org
> >> > > > > > > > >wrote:
> >> > > > > > > >
> >> > > > > > > > > Jasmine does support asynchronous tests, and waiting an
> >> > > arbitrary
> >> > > > > > > period
> >> > > > > > > > or
> >> > > > > > > > > until some condition is true. Why does that not work for
> >> > these
> >> > > > > tests?
> >> > > > > > > > >
> >> > > > > > > > > Braden
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <
> >> > > > > > amcharbo@gmail.com
> >> > > > > > > > > >wrote:
> >> > > > > > > > >
> >> > > > > > > > > > Quick follow up question for the return values stuff:
> >> > > > > > > > > > Does it make sense to have my functions called from
> >> execute
> >> > > > > > > (capture()
> >> > > > > > > > > and
> >> > > > > > > > > > captureAndCompare()) return an error string if one is
> >> > > > encountered
> >> > > > > > and
> >> > > > > > > > > pass
> >> > > > > > > > > > that back in callbackContext.error, then have execute
> >> > return
> >> > > > > true?
> >> > > > > > > > > >
> >> > > > > > > > > > Or is there a preferred way to get errors from sub
> >> > functions
> >> > > > back
> >> > > > > > > into
> >> > > > > > > > > > Javascript?
> >> > > > > > > > > >
> >> > > > > > > > > > Thanks!
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <
> >> > > > > > > > amcharbo@gmail.com
> >> > > > > > > > > > >wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > Thanks Andrew these are great suggestions!
> >> > > > > > > > > > >
> >> > > > > > > > > > > About not needing the busy-wait in getScreenBits:
> >> > > > > > > > > > > I'm not a fan of busy-waiting either.
> >> > > > > > > > > > > Perhaps it's a side effect of Jasmine requiring
> tests
> >> to
> >> > be
> >> > > > run
> >> > > > > > and
> >> > > > > > > > > > > evaluated synchronously that I decided to use a
> >> busy-wait
> >> > > in
> >> > > > > > > > > > getScreenBits.
> >> > > > > > > > > > > If the Runnable calls CallbackContext.success/error
> >> once
> >> > > the
> >> > > > > > bits
> >> > > > > > > > are
> >> > > > > > > > > > > captured (but not written to file) on the UIThread,
> >> the
> >> > > > > > Javascript
> >> > > > > > > > side
> >> > > > > > > > > > > will proceed to run before it has an image
> url/compare
> >> > > > result,
> >> > > > > > most
> >> > > > > > > > > > likely
> >> > > > > > > > > > > causing that test to fail. As far as I know Jasmine
> >> > > doesn't
> >> > > > > > allow
> >> > > > > > > > for
> >> > > > > > > > > > > kicking off a bunch of tests and then gather all the
> >> > > results
> >> > > > at
> >> > > > > > the
> >> > > > > > > > end
> >> > > > > > > > > > and
> >> > > > > > > > > > > output pass/fails (each 'expect' function needs to
> be
> >> > > within
> >> > > > an
> >> > > > > > > 'it'
> >> > > > > > > > > > block,
> >> > > > > > > > > > > essentially that means one test must be evaluated
> >> before
> >> > > the
> >> > > > > next
> >> > > > > > > can
> >> > > > > > > > > > > begin).
> >> > > > > > > > > > >
> >> > > > > > > > > > > I can see the usefulness of allowing the capture,
> file
> >> > io,
> >> > > > and
> >> > > > > > > > > comparison
> >> > > > > > > > > > > to be able to run asynchronously from the Javascript
> >> > > though.
> >> > > > In
> >> > > > > > the
> >> > > > > > > > > case
> >> > > > > > > > > > > where you have your own test framework that CAN kick
> >> off
> >> > a
> >> > > > > bunch
> >> > > > > > of
> >> > > > > > > > > tests
> >> > > > > > > > > > > and just gather the results as they come in and do a
> >> > batch
> >> > > > > > > evaluation
> >> > > > > > > > > at
> >> > > > > > > > > > > the end, or in the case where you don't want to do
> any
> >> > > > > evaluation
> >> > > > > > > in
> >> > > > > > > > > your
> >> > > > > > > > > > > app and just want to get captures as fast as you
> can,
> >> > > > > > asynchronous
> >> > > > > > > > > could
> >> > > > > > > > > > > increase performance there.
> >> > > > > > > > > > >
> >> > > > > > > > > > > I could try to implement it this way too, if you
> think
> >> > > > that's a
> >> > > > > > > worth
> >> > > > > > > > > > > while use case, maybe just provide an 'asynchronous'
> >> flag
> >> > > in
> >> > > > > > > > > > CaptureOptions?
> >> > > > > > > > > > >
> >> > > > > > > > > > > For doing the fileIO/compare using
> >> cordova.getThreadPool,
> >> > > is
> >> > > > > > there
> >> > > > > > > a
> >> > > > > > > > > > > benefit to doing that if I still plan having the
> >> > operation
> >> > > > > block
> >> > > > > > > the
> >> > > > > > > > > > > Javascript? For asynchronous mode I see why that
> >> makes
> >> > > > sense.
> >> > > > > > > > > > >
> >> > > > > > > > > > > For the return values, I'll make those changes right
> >> > away!
> >> > > > > > Thanks
> >> > > > > > > > > again
> >> > > > > > > > > > > for providing your insight, hopefully I understood
> it
> >> > > > > correctly.
> >> > > > > > > > > > >
> >> > > > > > > > > > > -Aaron
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <
> >> > > > > > > agrieve@chromium.org
> >> > > > > > > > > > >wrote:
> >> > > > > > > > > > >
> >> > > > > > > > > > >> Had a glance at your code. Things you may want to
> >> > address:
> >> > > > > > > > > > >>
> >> > > > > > > > > > >> while(!mUIThreadDone) {}
> >> > > > > > > > > > >>
> >> > > > > > > > > > >> You shouldn't need to busy-wait. Pass the
> >> > CallbackContext
> >> > > to
> >> > > > > the
> >> > > > > > > > > > Runnable,
> >> > > > > > > > > > >> and call success/error on it whenever you're done.
> No
> >> > need
> >> > > > to
> >> > > > > > have
> >> > > > > > > > the
> >> > > > > > > > > > web
> >> > > > > > > > > > >> thread wait for it. Even better would be to use a
> >> second
> >> > > > > > Runnable
> >> > > > > > > to
> >> > > > > > > > > do
> >> > > > > > > > > > >> the
> >> > > > > > > > > > >> image compare / file IO on a background thread via
> >> > > > > > > > > > cordova.getThreadPool()
> >> > > > > > > > > > >>
> >> > > > > > > > > > >> The return value to execute() should only be false
> >> if an
> >> > > > > invalid
> >> > > > > > > > > command
> >> > > > > > > > > > >> was passed. It's not meant to indicate the success
> of
> >> > the
> >> > > > > > > operation.
> >> > > > > > > > > You
> >> > > > > > > > > > >> should be returning true for it.
> >> > > > > > > > > > >>
> >> > > > > > > > > > >> Instead of return "success"/"", it might simplify
> >> > things a
> >> > > > bit
> >> > > > > > to
> >> > > > > > > > > > return a
> >> > > > > > > > > > >> boolean of true/false
> >> > > > > > > > > > >>
> >> > > > > > > > > > >>
> >> > > > > > > > > > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <
> >> > > > > > > > lorin.beer.dev@gmail.com
> >> > > > > > > > > > >> >wrote:
> >> > > > > > > > > > >>
> >> > > > > > > > > > >> > all the same, very impressive and faster then I
> >> > > expected.
> >> > > > > > > > > > >> >
> >> > > > > > > > > > >> > - Lorin
> >> > > > > > > > > > >> >
> >> > > > > > > > > > >> >
> >> > > > > > > > > > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron
> Charbonneau
> >> <
> >> > > > > > > > > > amcharbo@gmail.com
> >> > > > > > > > > > >> > >wrote:
> >> > > > > > > > > > >> >
> >> > > > > > > > > > >> > > Thanks Lorin.
> >> > > > > > > > > > >> > >
> >> > > > > > > > > > >> > > I did a quick test for throughput on an
> animating
> >> > > > canvas.
> >> > > > > I
> >> > > > > > > was
> >> > > > > > > > > > able
> >> > > > > > > > > > >> to
> >> > > > > > > > > > >> > > grab 30 frames within a 5 seconds which equates
> >> to
> >> > > > around
> >> > > > > > > 166ms
> >> > > > > > > > > per
> >> > > > > > > > > > >> > capture
> >> > > > > > > > > > >> > > on a Galaxy Nexus. Unfortunately not fast
> >> enough to
> >> > > > > > automate
> >> > > > > > > > > > >> something
> >> > > > > > > > > > >> > > like video. As with any kind of capturing
> >> system,
> >> > > often
> >> > > > > the
> >> > > > > > > > > > overhead
> >> > > > > > > > > > >> of
> >> > > > > > > > > > >> > > copying bits around is enough to slow down the
> >> > > operation
> >> > > > > of
> >> > > > > > > the
> >> > > > > > > > > app.
> >> > > > > > > > > > >> > >
> >> > > > > > > > > > >> > > -Aaron
> >> > > > > > > > > > >> > >
> >> > > > > > > > > > >> > >
> >> > > > > > > > > > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
> >> > > > > > > > > > >> lorin.beer.dev@gmail.com
> >> > > > > > > > > > >> > > >wrote:
> >> > > > > > > > > > >> > >
> >> > > > > > > > > > >> > > > Hey Aaron,
> >> > > > > > > > > > >> > > >
> >> > > > > > > > > > >> > > > very cool stuff, looking forward to checking
> it
> >> > out!
> >> > > > > > > > > > >> > > >
> >> > > > > > > > > > >> > > > Question: any performance hit on the app by
> >> using
> >> > > > > Capture?
> >> > > > > > > > > What's
> >> > > > > > > > > > >> the
> >> > > > > > > > > > >> > > > expected throughput on images taken in this
> way
> >> > > > (given a
> >> > > > > > > > > > particular
> >> > > > > > > > > > >> > > device,
> >> > > > > > > > > > >> > > > say Galaxy Nexus)?
> >> > > > > > > > > > >> > > >
> >> > > > > > > > > > >> > > > - Lorin
> >> > > > > > > > > > >> > > >
> >> > > > > > > > > > >> > > >
> >> > > > > > > > > > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron
> >> > Charbonneau
> >> > > <
> >> > > > > > > > > > >> > amcharbo@gmail.com
> >> > > > > > > > > > >> > > > >wrote:
> >> > > > > > > > > > >> > > >
> >> > > > > > > > > > >> > > > > Glad you like it :)
> >> > > > > > > > > > >> > > > > Yes in fact the actual capture makes use of
> >> > > > > > > > > > View.capturePicture()
> >> > > > > > > > > > >> > which
> >> > > > > > > > > > >> > > > > actually grabs the entire document, then
> that
> >> > can
> >> > > be
> >> > > > > > > clipped
> >> > > > > > > > > > down
> >> > > > > > > > > > >> to
> >> > > > > > > > > > >> > > the
> >> > > > > > > > > > >> > > > > size/location of a specific element.
> >> > > > > > > > > > >> > > > >
> >> > > > > > > > > > >> > > > > -Aaron
> >> > > > > > > > > > >> > > > >
> >> > > > > > > > > > >> > > > >
> >> > > > > > > > > > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal
> >> Mocny <
> >> > > > > > > > > > >> mmocny@chromium.org>
> >> > > > > > > > > > >> > > > > wrote:
> >> > > > > > > > > > >> > > > >
> >> > > > > > > > > > >> > > > > > Aaron,
> >> > > > > > > > > > >> > > > > >
> >> > > > > > > > > > >> > > > > > I haven't even begun looking at your
> >> > > > implementation,
> >> > > > > > but
> >> > > > > > > > I'm
> >> > > > > > > > > > >> just
> >> > > > > > > > > > >> > > going
> >> > > > > > > > > > >> > > > > to
> >> > > > > > > > > > >> > > > > > say it: this is awesome!
> >> > > > > > > > > > >> > > > > >
> >> > > > > > > > > > >> > > > > > First question: When Capturing a DOM
> >> element,
> >> > > can
> >> > > > > you
> >> > > > > > > > > capture
> >> > > > > > > > > > >> > 'body'
> >> > > > > > > > > > >> > > to
> >> > > > > > > > > > >> > > > > > grab it&children for a "full content
> >> > > screenshot",
> >> > > > or
> >> > > > > > > does
> >> > > > > > > > it
> >> > > > > > > > > > >> have
> >> > > > > > > > > > >> > to
> >> > > > > > > > > > >> > > > be a
> >> > > > > > > > > > >> > > > > > specific single element?
> >> > > > > > > > > > >> > > > > >
> >> > > > > > > > > > >> > > > > > -Michal
> >> > > > > > > > > > >> > > > > >
> >> > > > > > > > > > >> > > > > >
> >> > > > > > > > > > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron
> >> > > > Charbonneau <
> >> > > > > > > > > > >> > > amcharbo@gmail.com
> >> > > > > > > > > > >> > > > > > >wrote:
> >> > > > > > > > > > >> > > > > >
> >> > > > > > > > > > >> > > > > > > Greetings! My name is Aaron
> Charbonneau,
> >> > > happy
> >> > > > to
> >> > > > > > be
> >> > > > > > > a
> >> > > > > > > > > new
> >> > > > > > > > > > >> > member
> >> > > > > > > > > > >> > > of
> >> > > > > > > > > > >> > > > > the
> >> > > > > > > > > > >> > > > > > > mailing list!
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > > I have been developing a sceenshot
> plugin
> >> > for
> >> > > > > > Cordova
> >> > > > > > > to
> >> > > > > > > > > > help
> >> > > > > > > > > > >> > > > > facilitate
> >> > > > > > > > > > >> > > > > > > automation testing and debugging of
> >> Cordova
> >> > > > apps,
> >> > > > > > and
> >> > > > > > > I
> >> > > > > > > > > > would
> >> > > > > > > > > > >> > love
> >> > > > > > > > > > >> > > > some
> >> > > > > > > > > > >> > > > > > > feedback on it. Currently Cordova
> >> provides
> >> > a
> >> > > > > bunch
> >> > > > > > of
> >> > > > > > > > > > native
> >> > > > > > > > > > >> > > > functions
> >> > > > > > > > > > >> > > > > > > that allow you to do some cool stuff,
> but
> >> > not
> >> > > > much
> >> > > > > > > > > > >> functionality
> >> > > > > > > > > > >> > to
> >> > > > > > > > > > >> > > > > test
> >> > > > > > > > > > >> > > > > > > the apps that make use of them. Being
> >> able
> >> > to
> >> > > > > take
> >> > > > > > a
> >> > > > > > > > > > capture
> >> > > > > > > > > > >> of
> >> > > > > > > > > > >> > > the
> >> > > > > > > > > > >> > > > > > screen
> >> > > > > > > > > > >> > > > > > > from within you app is a great way to
> >> > automate
> >> > > > > > testing
> >> > > > > > > > or
> >> > > > > > > > > > get
> >> > > > > > > > > > >> > > > > additional
> >> > > > > > > > > > >> > > > > > > information for debugging an issue.
> >> Since
> >> > > there
> >> > > > > is
> >> > > > > > no
> >> > > > > > > > > > >> > > > > > > Javascript mechanism for taking screen
> >> > > captures
> >> > > > > the
> >> > > > > > > > > solution
> >> > > > > > > > > > >> > would
> >> > > > > > > > > > >> > > > have
> >> > > > > > > > > > >> > > > > > to
> >> > > > > > > > > > >> > > > > > > be native, which fits nicely into the
> >> "gap"
> >> > > that
> >> > > > > > > > > > >> Cordova/Phonegap
> >> > > > > > > > > > >> > > > > > bridges.
> >> > > > > > > > > > >> > > > > > > Any medium to large scale app can
> >> benefit
> >> > > > greatly
> >> > > > > > > from
> >> > > > > > > > > > >> > automation
> >> > > > > > > > > > >> > > > > > testing
> >> > > > > > > > > > >> > > > > > > and any app can benefit from an extra
> >> > > debugging
> >> > > > > > tool,
> >> > > > > > > > and
> >> > > > > > > > > > >> that is
> >> > > > > > > > > > >> > > > what
> >> > > > > > > > > > >> > > > > I
> >> > > > > > > > > > >> > > > > > > hope this screenshot plugin can help
> >> > achieve.
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > > Currently the plugin offers 2
> functions:
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > > Capture():
> >> > > > > > > > > > >> > > > > > > * Take a capture of the current view,
> >> write
> >> > > that
> >> > > > > > > capture
> >> > > > > > > > > to
> >> > > > > > > > > > a
> >> > > > > > > > > > >> > .png
> >> > > > > > > > > > >> > > > file
> >> > > > > > > > > > >> > > > > > > with the specified file name and sub
> >> > directory
> >> > > > of
> >> > > > > > the
> >> > > > > > > > > sdcard
> >> > > > > > > > > > >> > > > (fallback
> >> > > > > > > > > > >> > > > > to
> >> > > > > > > > > > >> > > > > > > emulated sdcard in the case there isn't
> >> an
> >> > > > sdcard
> >> > > > > > > > mounted)
> >> > > > > > > > > > >> > > > > > > * Able to create a sub-screenshot with
> a
> >> > > > specified
> >> > > > > > > > > rectangle
> >> > > > > > > > > > >> in
> >> > > > > > > > > > >> > > order
> >> > > > > > > > > > >> > > > > to
> >> > > > > > > > > > >> > > > > > > block out ui elements that may be
> >> variable,
> >> > > and
> >> > > > > also
> >> > > > > > > > save
> >> > > > > > > > > > >> space.
> >> > > > > > > > > > >> > > > > > > * Can take captures of images/dom
> >> elements
> >> > > > > > (including
> >> > > > > > > > > > canvas)
> >> > > > > > > > > > >> > that
> >> > > > > > > > > > >> > > > are
> >> > > > > > > > > > >> > > > > > > lager than the actual screen size
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > > CaptureAndCompare():
> >> > > > > > > > > > >> > > > > > > * All the functionality of Capture()
> >> > > > > > > > > > >> > > > > > > * Perform a comparison between the
> >> captured
> >> > > > image
> >> > > > > > and
> >> > > > > > > a
> >> > > > > > > > > > >> baseline
> >> > > > > > > > > > >> > > > image
> >> > > > > > > > > > >> > > > > > > located at the specified location in
> >> either
> >> > > the
> >> > > > > > assets
> >> > > > > > > > > > folder
> >> > > > > > > > > > >> or
> >> > > > > > > > > > >> > > the
> >> > > > > > > > > > >> > > > > > > sdcard.
> >> > > > > > > > > > >> > > > > > > * User can specify per color channel
> >> > > tolerances
> >> > > > as
> >> > > > > > > well
> >> > > > > > > > as
> >> > > > > > > > > > >> total
> >> > > > > > > > > > >> > > > pixel
> >> > > > > > > > > > >> > > > > > > tolerances to avoid false positives for
> >> the
> >> > > > > > inevitable
> >> > > > > > > > > > >> rendering
> >> > > > > > > > > > >> > > > > > > differences across devices.
> >> > > > > > > > > > >> > > > > > > * Optionally output a png file that
> >> contains
> >> > > the
> >> > > > > > > > > differences
> >> > > > > > > > > > >> > > between
> >> > > > > > > > > > >> > > > > the
> >> > > > > > > > > > >> > > > > > > actual and the baseline for
> >> debugging/triage
> >> > > > > > purposes,
> >> > > > > > > > two
> >> > > > > > > > > > >> modes:
> >> > > > > > > > > > >> > > > > binary
> >> > > > > > > > > > >> > > > > > > diff (all failing pixels appear as
> solid
> >> > > white)
> >> > > > or
> >> > > > > > the
> >> > > > > > > > > true
> >> > > > > > > > > > >> > > > differences
> >> > > > > > > > > > >> > > > > > > between pixels.
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > > If you can spare some time, I would
> love
> >> it
> >> > if
> >> > > > you
> >> > > > > > > could
> >> > > > > > > > > > take
> >> > > > > > > > > > >> a
> >> > > > > > > > > > >> > > look
> >> > > > > > > > > > >> > > > at
> >> > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > >> > > > > > > api and parameters I have defined to
> make
> >> > sure
> >> > > > > they
> >> > > > > > > > adhere
> >> > > > > > > > > > to
> >> > > > > > > > > > >> > > Cordova
> >> > > > > > > > > > >> > > > > > > plugin best practices. The most
> crucial
> >> > part
> >> > > > > would
> >> > > > > > be
> >> > > > > > > > in
> >> > > > > > > > > > the
> >> > > > > > > > > > >> > > plugin
> >> > > > > > > > > > >> > > > > > itself
> >> > > > > > > > > > >> > > > > > > at ScreenCapture.java:
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > >
> >> > > > > > > > > > >> > > > >
> >> > > > > > > > > > >> > > >
> >> > > > > > > > > > >> > >
> >> > > > > > > > > > >> >
> >> > > > > > > > > > >>
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> >> > > > > > > > > > >> > > > > > > and captureAndCompare() are the two
> >> exposed
> >> > > > > > functions)
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > > I'm also interested to know you
> thoughts
> >> on
> >> > > it's
> >> > > > > > > usage.
> >> > > > > > > > I
> >> > > > > > > > > > >> > started
> >> > > > > > > > > > >> > > > with
> >> > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > >> > > > > > > mobile-spec testing framework and put
> >> some
> >> > > quick
> >> > > > > > > > rendering
> >> > > > > > > > > > >> tests
> >> > > > > > > > > > >> > > into
> >> > > > > > > > > > >> > > > > the
> >> > > > > > > > > > >> > > > > > > Autotest section as a viable use case.
> >> In
> >> > > order
> >> > > > > to
> >> > > > > > > get
> >> > > > > > > > > the
> >> > > > > > > > > > >> > WebView
> >> > > > > > > > > > >> > > > > into
> >> > > > > > > > > > >> > > > > > > the state I wanted to capture, I had to
> >> > > > implement
> >> > > > > a
> >> > > > > > > wait
> >> > > > > > > > > > >> timer to
> >> > > > > > > > > > >> > > > allow
> >> > > > > > > > > > >> > > > > > the
> >> > > > > > > > > > >> > > > > > > view to update before taking the
> capture.
> >> > > Once
> >> > > > > the
> >> > > > > > > wait
> >> > > > > > > > > > >> function
> >> > > > > > > > > > >> > > > > > > completes, the capture can be taken and
> >> > > > everything
> >> > > > > > > from
> >> > > > > > > > > > there
> >> > > > > > > > > > >> is
> >> > > > > > > > > > >> > > > > callback
> >> > > > > > > > > > >> > > > > > > based. I use the Jasmine
> >> waitsFor()/runs()
> >> > > > blocks
> >> > > > > > to
> >> > > > > > > > make
> >> > > > > > > > > > the
> >> > > > > > > > > > >> > > tests
> >> > > > > > > > > > >> > > > > run
> >> > > > > > > > > > >> > > > > > > synchronously. For some validation
> >> testing
> >> > I
> >> > > > > made a
> >> > > > > > > > > compare
> >> > > > > > > > > > >> > > function
> >> > > > > > > > > > >> > > > > in
> >> > > > > > > > > > >> > > > > > > Javascript to run against the native
> >> > compare.
> >> > > > > Turns
> >> > > > > > > out
> >> > > > > > > > > > Java
> >> > > > > > > > > > >> > runs
> >> > > > > > > > > > >> > > > alot
> >> > > > > > > > > > >> > > > > > > faster than Javascript (surprise!) thus
> >> the
> >> > > > native
> >> > > > > > > > compare
> >> > > > > > > > > > is
> >> > > > > > > > > > >> > about
> >> > > > > > > > > > >> > > > > 2-5x
> >> > > > > > > > > > >> > > > > > > faster. All in all the process is
> fairly
> >> > > quick,
> >> > > > > the
> >> > > > > > > > full
> >> > > > > > > > > > >> > > > > > captureAndCompare
> >> > > > > > > > > > >> > > > > > > with 3 file io's takes about 233ms on a
> >> > Nexus
> >> > > > One,
> >> > > > > > and
> >> > > > > > > > > > ~100ms
> >> > > > > > > > > > >> on
> >> > > > > > > > > > >> > a
> >> > > > > > > > > > >> > > > > Nexus
> >> > > > > > > > > > >> > > > > > 10
> >> > > > > > > > > > >> > > > > > > (256x256 image size).
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > > Anyways here is the usage:
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > >
> >> > > > > > > > > > >> > > > >
> >> > > > > > > > > > >> > > >
> >> > > > > > > > > > >> > >
> >> > > > > > > > > > >> >
> >> > > > > > > > > > >>
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > > And here's the JS wrapper for the
> plugin
> >> > > calls:
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > >
> >> > > > > > > > > > >> > > > >
> >> > > > > > > > > > >> > > >
> >> > > > > > > > > > >> > >
> >> > > > > > > > > > >> >
> >> > > > > > > > > > >>
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > > Thanks for your time, and I look
> forward
> >> to
> >> > > ANY
> >> > > > > > > feedback
> >> > > > > > > > > > >> positive
> >> > > > > > > > > > >> > > or
> >> > > > > > > > > > >> > > > > > > negative.
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > > > Thanks,
> >> > > > > > > > > > >> > > > > > > Aaron
> >> > > > > > > > > > >> > > > > > >
> >> > > > > > > > > > >> > > > > >
> >> > > > > > > > > > >> > > > >
> >> > > > > > > > > > >> > > >
> >> > > > > > > > > > >> > >
> >> > > > > > > > > > >> >
> >> > > > > > > > > > >>
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Aaron Charbonneau <am...@gmail.com>.
Ok thought this was all closed out, but I've hit a little snag with the
'spawn background thread from ui thread'.
For some reason for the throughput case, spawning the fileIO background
thread directly from the ui thread causes the ui to become much less fluid,
causing more jerking in a simple animation than spawning the fileIO thread
from the WebCore thread. The code is almost identical between the 2
methods, so it seems odd this would be happening. My only thought is that
the ui thread now has to hold on to some references of that background
thread? Perplexing, and not easy to find info on the interwebs about this.
Any thoughts?
-Aaron
On Mon, Mar 18, 2013 at 1:57 PM, Aaron Charbonneau <am...@gmail.com>wrote:
> Cool thanks for explaining that, alot of the information for Cordova
> plugin dev is a little out-dated, makes it hard sometimes to 'do the right
> thing.'
>
> Ok I've made all the changes:
>
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
>
> I'll probably go forward with this approach once I've written some more
> tests to make sure there's no issues, and test out the additional
> functionality for throughput captures.
>
> Thanks again the help and feedback, was a very good learning experience.
> Please let me know if you see any other issues!
>
> Thanks,
> Aaron
>
>
> On Mon, Mar 18, 2013 at 1:44 PM, Andrew Grieve <ag...@chromium.org>wrote:
>
>> On Mon, Mar 18, 2013 at 4:38 PM, Aaron Charbonneau <amcharbo@gmail.com
>> >wrote:
>>
>> > "You could avoid the blocking queue if you dispatch the background
>> thread's
>> > runnable at the end of the UI thread's block."
>> >
>> > -Great idea, done.
>> >
>> >
>> > I traced through to see what sending a NO_RESULT PluginResult with
>> > keepCallback==true does, and it looks like a no-op
>> > (NativeToJsMessageQueue.java, addPluginResult()):
>> >
>> > boolean noResult = result.getStatus() ==
>> > PluginResult.Status.NO_RESULT.ordinal();
>> > boolean keepCallback = result.getKeepCallback();
>> > if (noResult && keepCallback) {
>> > return;
>> > }
>> >
>> > I'm sure I'm missing something here, is that really a no-op?
>>
>> Yep, that's a no-op and exists only to support the previous style of
>> Android plugins, where the execute method *had* to return a PluginResult.
>>
>>
>>
>> > Did you
>> > mean for me to send a PluginResult.Status.OK with keepCallBack=true?
>> >
>> That's exactly what I meant :)
>>
>>
>> >
>> >
>> > Thanks,
>> > Aaron
>> >
>> >
>> >
>> >
>> > On Sat, Mar 16, 2013 at 6:15 AM, Andrew Grieve <ag...@chromium.org>
>> > wrote:
>> >
>> > > You could avoid the blocking queue if you dispatch the background
>> > thread's
>> > > runnable at the end of the UI thread's block.
>> > >
>> > > I see you send a success callback as soon as the image is captured,
>> but
>> > > before it is saved. This would improve throughput, but it might be
>> even
>> > > more useful if you set the keepCallback flag on it and then send
>> another
>> > > success response once you have the filename / comparison results. It's
>> > the
>> > > filename that the app is probably interested in.
>> > >
>> > >
>> > > On Fri, Mar 15, 2013 at 9:41 PM, Aaron Charbonneau <
>> amcharbo@gmail.com
>> > > >wrote:
>> > >
>> > > > Ah thanks for clarifying that.
>> > > >
>> > > > I'm not sure how I can alleviate the ui thread any more than I am
>> > > > currently. It is already doing the bare minimum amount of work on
>> the
>> > > > there. Right now the only work done on the ui thread is
>> > > > view.capturePicture() which is the recommended thread for doing that
>> > work
>> > > > (otherwise you get a warning):
>> > > >
>> > > >
>> > > > mUIThreadDone = false;
>> > > > cordova.getActivity().runOnUiThread(new Runnable() {
>> > > > public void run() {
>> > > > CordovaWebView uiThreadView = webView;
>> > > > picture = uiThreadView.capturePicture();
>> > > > mUIThreadDone = true;
>> > > > }
>> > > > });
>> > > > //rest of the work is done on core thread
>> > > > while(!mUIThreadDone) {}
>> > > > //write the picture to a file
>> > > > ...
>> > > >
>> > > > I could do the file io and compares on a different thread than the
>> > > > core one, but for the major use case, the core thread needs to wait
>> no
>> > > > matter what to ensure that the file is written and the comparison is
>> > > > complete, so I wouldn't expect much gain there.
>> > > >
>> > > >
>> > > > However, I've been experimenting with a more threaded version that
>> > > > does the file io and comparison on a background thread as you
>> > > > suggested. If you have time please check it out. While the code is
>> > > > not as clean, I think it addresses the issues you raised Andrew.
>> For
>> > > > the major use case, performance is still the same as expected, but
>> if
>> > > > we want pure throughput on captures there's a big gain from about 3
>> > > > captures per second (cps) to 10 cps on a Nexus 10.
>> > > >
>> > > >
>> > > > Here's the new plugin, with moar threading!
>> > > >
>> > > >
>> > > >
>> > > >
>> > >
>> >
>> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
>> > > >
>> > > >
>> > > > Once again, thanks for all your feedback on this, I haven't had much
>> > > > experience with threading in java before now.
>> > > >
>> > > >
>> > > > -Aaron
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Thu, Mar 14, 2013 at 8:10 AM, Andrew Grieve <
>> agrieve@chromium.org>
>> > > > wrote:
>> > > >
>> > > > > My suggestion was just to prevent the UI thread from locking up so
>> > that
>> > > > you
>> > > > > app doesn't appear to be frozen when the plugin is doing its
>> thing.
>> > > > >
>> > > > >
>> > > > > On Wed, Mar 13, 2013 at 5:03 PM, Aaron Charbonneau <
>> > amcharbo@gmail.com
>> > > > > >wrote:
>> > > > >
>> > > > > > Yah that's how I have it working now :) No problem with that
>> > approach
>> > > > for
>> > > > > > working with Jasmine.
>> > > > > >
>> > > > > > I made reference to running them asynchronously to respond to
>> > Andrew
>> > > > > about
>> > > > > > returning immediately after the bits are captured, which in the
>> > case
>> > > of
>> > > > > > using that with Jasmine would only see a benefit IF we could run
>> > > > multiple
>> > > > > > 'it' blocks at the same time.
>> > > > > >
>> > > > > > I think however Andrew was thinking of a different use case
>> where
>> > we
>> > > > > aren't
>> > > > > > limited by a framework that requires tests to run serially, and
>> > > results
>> > > > > can
>> > > > > > just come back as they are ready, and be evaluated all together
>> at
>> > > the
>> > > > > end,
>> > > > > > or not at all if you just want the captures and no evaluation.
>> > > > > >
>> > > > > > -Aaron
>> > > > > >
>> > > > > >
>> > > > > > On Wed, Mar 13, 2013 at 1:53 PM, Braden Shepherdson <
>> > > > braden@chromium.org
>> > > > > > >wrote:
>> > > > > >
>> > > > > > > As far as I know, you're correct: there's no way to move on
>> the
>> > > next
>> > > > > it()
>> > > > > > > block before the first one is done.
>> > > > > > >
>> > > > > > > What's the problem with just making the tests wait? Why does
>> the
>> > > next
>> > > > > > it()
>> > > > > > > block need to start early? You can make your call to Java, use
>> > > > waitFor
>> > > > > to
>> > > > > > > wait until the callback with the data is called, run your
>> > > > expectations,
>> > > > > > and
>> > > > > > > then move on to the next it() block. If the next it() block is
>> > > using
>> > > > > the
>> > > > > > > result of this one, then they should be one it() block.
>> > > > > > >
>> > > > > > > Braden
>> > > > > > >
>> > > > > > >
>> > > > > > > On Wed, Mar 13, 2013 at 4:46 PM, Aaron Charbonneau <
>> > > > amcharbo@gmail.com
>> > > > > > > >wrote:
>> > > > > > >
>> > > > > > > > Hi Braden,
>> > > > > > > > Sorry maybe I am overlooking something. The way I
>> understand
>> > it
>> > > is
>> > > > > > that
>> > > > > > > a
>> > > > > > > > previous 'it' block needs to complete before the next one
>> can
>> > > run.
>> > > > > So
>> > > > > > I
>> > > > > > > > have various tests setup like this:
>> > > > > > > >
>> > > > > > > > it("renders something", function() {
>> > > > > > > > //draw something
>> > > > > > > > ....
>> > > > > > > > captureAndCompare(); //call the native function
>> > > > > > > > waitsFor(function() {
>> > > > > > > > return captureComplete; //set by the native function
>> callback
>> > > > > > > > }, "capture never completed", 10000);
>> > > > > > > > runs(function() {
>> > > > > > > > expect(captureResult).toBe(0); //expects difference to be 0
>> > > > > > > > });
>> > > > > > > > });
>> > > > > > > > //next 'it' test
>> > > > > > > > ...
>> > > > > > > >
>> > > > > > > > The expect block needs to wait until there is a result
>> returned
>> > > > from
>> > > > > > the
>> > > > > > > > native captureAndCompare call. Therefore we use a
>> > waitsFor/runs
>> > > > > block
>> > > > > > > > there.
>> > > > > > > >
>> > > > > > > > But say we wanted to allow Javascript to run immediately
>> after
>> > > > > calling
>> > > > > > > that
>> > > > > > > > native function. The only thing you need to wait for is
>> that
>> > > Java
>> > > > > > > grabbed
>> > > > > > > > the bits, and the rest of the processing will be done 'at
>> some
>> > > > point
>> > > > > in
>> > > > > > > the
>> > > > > > > > future.' So as Andrew suggested you would return as soon as
>> > the
>> > > > bits
>> > > > > > are
>> > > > > > > > captured, and allow JS to run again. The problem is that
>> our
>> > > > expect
>> > > > > > > block
>> > > > > > > > needs the results, so that has to wait, the only other
>> thing to
>> > > do
>> > > > > > would
>> > > > > > > be
>> > > > > > > > to run the next 'it' block.
>> > > > > > > >
>> > > > > > > > This is the part I didn't think was possible, can the next
>> 'it'
>> > > > block
>> > > > > > be
>> > > > > > > > run while the previous 'it' is still waiting to execute the
>> > > > 'expect'
>> > > > > > > block?
>> > > > > > > > Or is there another approach all together that would allow
>> > this?
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > Aaron
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <
>> > > > > > braden@chromium.org
>> > > > > > > > >wrote:
>> > > > > > > >
>> > > > > > > > > Jasmine does support asynchronous tests, and waiting an
>> > > arbitrary
>> > > > > > > period
>> > > > > > > > or
>> > > > > > > > > until some condition is true. Why does that not work for
>> > these
>> > > > > tests?
>> > > > > > > > >
>> > > > > > > > > Braden
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <
>> > > > > > amcharbo@gmail.com
>> > > > > > > > > >wrote:
>> > > > > > > > >
>> > > > > > > > > > Quick follow up question for the return values stuff:
>> > > > > > > > > > Does it make sense to have my functions called from
>> execute
>> > > > > > > (capture()
>> > > > > > > > > and
>> > > > > > > > > > captureAndCompare()) return an error string if one is
>> > > > encountered
>> > > > > > and
>> > > > > > > > > pass
>> > > > > > > > > > that back in callbackContext.error, then have execute
>> > return
>> > > > > true?
>> > > > > > > > > >
>> > > > > > > > > > Or is there a preferred way to get errors from sub
>> > functions
>> > > > back
>> > > > > > > into
>> > > > > > > > > > Javascript?
>> > > > > > > > > >
>> > > > > > > > > > Thanks!
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <
>> > > > > > > > amcharbo@gmail.com
>> > > > > > > > > > >wrote:
>> > > > > > > > > >
>> > > > > > > > > > > Thanks Andrew these are great suggestions!
>> > > > > > > > > > >
>> > > > > > > > > > > About not needing the busy-wait in getScreenBits:
>> > > > > > > > > > > I'm not a fan of busy-waiting either.
>> > > > > > > > > > > Perhaps it's a side effect of Jasmine requiring tests
>> to
>> > be
>> > > > run
>> > > > > > and
>> > > > > > > > > > > evaluated synchronously that I decided to use a
>> busy-wait
>> > > in
>> > > > > > > > > > getScreenBits.
>> > > > > > > > > > > If the Runnable calls CallbackContext.success/error
>> once
>> > > the
>> > > > > > bits
>> > > > > > > > are
>> > > > > > > > > > > captured (but not written to file) on the UIThread,
>> the
>> > > > > > Javascript
>> > > > > > > > side
>> > > > > > > > > > > will proceed to run before it has an image url/compare
>> > > > result,
>> > > > > > most
>> > > > > > > > > > likely
>> > > > > > > > > > > causing that test to fail. As far as I know Jasmine
>> > > doesn't
>> > > > > > allow
>> > > > > > > > for
>> > > > > > > > > > > kicking off a bunch of tests and then gather all the
>> > > results
>> > > > at
>> > > > > > the
>> > > > > > > > end
>> > > > > > > > > > and
>> > > > > > > > > > > output pass/fails (each 'expect' function needs to be
>> > > within
>> > > > an
>> > > > > > > 'it'
>> > > > > > > > > > block,
>> > > > > > > > > > > essentially that means one test must be evaluated
>> before
>> > > the
>> > > > > next
>> > > > > > > can
>> > > > > > > > > > > begin).
>> > > > > > > > > > >
>> > > > > > > > > > > I can see the usefulness of allowing the capture, file
>> > io,
>> > > > and
>> > > > > > > > > comparison
>> > > > > > > > > > > to be able to run asynchronously from the Javascript
>> > > though.
>> > > > In
>> > > > > > the
>> > > > > > > > > case
>> > > > > > > > > > > where you have your own test framework that CAN kick
>> off
>> > a
>> > > > > bunch
>> > > > > > of
>> > > > > > > > > tests
>> > > > > > > > > > > and just gather the results as they come in and do a
>> > batch
>> > > > > > > evaluation
>> > > > > > > > > at
>> > > > > > > > > > > the end, or in the case where you don't want to do any
>> > > > > evaluation
>> > > > > > > in
>> > > > > > > > > your
>> > > > > > > > > > > app and just want to get captures as fast as you can,
>> > > > > > asynchronous
>> > > > > > > > > could
>> > > > > > > > > > > increase performance there.
>> > > > > > > > > > >
>> > > > > > > > > > > I could try to implement it this way too, if you think
>> > > > that's a
>> > > > > > > worth
>> > > > > > > > > > > while use case, maybe just provide an 'asynchronous'
>> flag
>> > > in
>> > > > > > > > > > CaptureOptions?
>> > > > > > > > > > >
>> > > > > > > > > > > For doing the fileIO/compare using
>> cordova.getThreadPool,
>> > > is
>> > > > > > there
>> > > > > > > a
>> > > > > > > > > > > benefit to doing that if I still plan having the
>> > operation
>> > > > > block
>> > > > > > > the
>> > > > > > > > > > > Javascript? For asynchronous mode I see why that
>> makes
>> > > > sense.
>> > > > > > > > > > >
>> > > > > > > > > > > For the return values, I'll make those changes right
>> > away!
>> > > > > > Thanks
>> > > > > > > > > again
>> > > > > > > > > > > for providing your insight, hopefully I understood it
>> > > > > correctly.
>> > > > > > > > > > >
>> > > > > > > > > > > -Aaron
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <
>> > > > > > > agrieve@chromium.org
>> > > > > > > > > > >wrote:
>> > > > > > > > > > >
>> > > > > > > > > > >> Had a glance at your code. Things you may want to
>> > address:
>> > > > > > > > > > >>
>> > > > > > > > > > >> while(!mUIThreadDone) {}
>> > > > > > > > > > >>
>> > > > > > > > > > >> You shouldn't need to busy-wait. Pass the
>> > CallbackContext
>> > > to
>> > > > > the
>> > > > > > > > > > Runnable,
>> > > > > > > > > > >> and call success/error on it whenever you're done. No
>> > need
>> > > > to
>> > > > > > have
>> > > > > > > > the
>> > > > > > > > > > web
>> > > > > > > > > > >> thread wait for it. Even better would be to use a
>> second
>> > > > > > Runnable
>> > > > > > > to
>> > > > > > > > > do
>> > > > > > > > > > >> the
>> > > > > > > > > > >> image compare / file IO on a background thread via
>> > > > > > > > > > cordova.getThreadPool()
>> > > > > > > > > > >>
>> > > > > > > > > > >> The return value to execute() should only be false
>> if an
>> > > > > invalid
>> > > > > > > > > command
>> > > > > > > > > > >> was passed. It's not meant to indicate the success of
>> > the
>> > > > > > > operation.
>> > > > > > > > > You
>> > > > > > > > > > >> should be returning true for it.
>> > > > > > > > > > >>
>> > > > > > > > > > >> Instead of return "success"/"", it might simplify
>> > things a
>> > > > bit
>> > > > > > to
>> > > > > > > > > > return a
>> > > > > > > > > > >> boolean of true/false
>> > > > > > > > > > >>
>> > > > > > > > > > >>
>> > > > > > > > > > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <
>> > > > > > > > lorin.beer.dev@gmail.com
>> > > > > > > > > > >> >wrote:
>> > > > > > > > > > >>
>> > > > > > > > > > >> > all the same, very impressive and faster then I
>> > > expected.
>> > > > > > > > > > >> >
>> > > > > > > > > > >> > - Lorin
>> > > > > > > > > > >> >
>> > > > > > > > > > >> >
>> > > > > > > > > > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau
>> <
>> > > > > > > > > > amcharbo@gmail.com
>> > > > > > > > > > >> > >wrote:
>> > > > > > > > > > >> >
>> > > > > > > > > > >> > > Thanks Lorin.
>> > > > > > > > > > >> > >
>> > > > > > > > > > >> > > I did a quick test for throughput on an animating
>> > > > canvas.
>> > > > > I
>> > > > > > > was
>> > > > > > > > > > able
>> > > > > > > > > > >> to
>> > > > > > > > > > >> > > grab 30 frames within a 5 seconds which equates
>> to
>> > > > around
>> > > > > > > 166ms
>> > > > > > > > > per
>> > > > > > > > > > >> > capture
>> > > > > > > > > > >> > > on a Galaxy Nexus. Unfortunately not fast
>> enough to
>> > > > > > automate
>> > > > > > > > > > >> something
>> > > > > > > > > > >> > > like video. As with any kind of capturing
>> system,
>> > > often
>> > > > > the
>> > > > > > > > > > overhead
>> > > > > > > > > > >> of
>> > > > > > > > > > >> > > copying bits around is enough to slow down the
>> > > operation
>> > > > > of
>> > > > > > > the
>> > > > > > > > > app.
>> > > > > > > > > > >> > >
>> > > > > > > > > > >> > > -Aaron
>> > > > > > > > > > >> > >
>> > > > > > > > > > >> > >
>> > > > > > > > > > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
>> > > > > > > > > > >> lorin.beer.dev@gmail.com
>> > > > > > > > > > >> > > >wrote:
>> > > > > > > > > > >> > >
>> > > > > > > > > > >> > > > Hey Aaron,
>> > > > > > > > > > >> > > >
>> > > > > > > > > > >> > > > very cool stuff, looking forward to checking it
>> > out!
>> > > > > > > > > > >> > > >
>> > > > > > > > > > >> > > > Question: any performance hit on the app by
>> using
>> > > > > Capture?
>> > > > > > > > > What's
>> > > > > > > > > > >> the
>> > > > > > > > > > >> > > > expected throughput on images taken in this way
>> > > > (given a
>> > > > > > > > > > particular
>> > > > > > > > > > >> > > device,
>> > > > > > > > > > >> > > > say Galaxy Nexus)?
>> > > > > > > > > > >> > > >
>> > > > > > > > > > >> > > > - Lorin
>> > > > > > > > > > >> > > >
>> > > > > > > > > > >> > > >
>> > > > > > > > > > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron
>> > Charbonneau
>> > > <
>> > > > > > > > > > >> > amcharbo@gmail.com
>> > > > > > > > > > >> > > > >wrote:
>> > > > > > > > > > >> > > >
>> > > > > > > > > > >> > > > > Glad you like it :)
>> > > > > > > > > > >> > > > > Yes in fact the actual capture makes use of
>> > > > > > > > > > View.capturePicture()
>> > > > > > > > > > >> > which
>> > > > > > > > > > >> > > > > actually grabs the entire document, then that
>> > can
>> > > be
>> > > > > > > clipped
>> > > > > > > > > > down
>> > > > > > > > > > >> to
>> > > > > > > > > > >> > > the
>> > > > > > > > > > >> > > > > size/location of a specific element.
>> > > > > > > > > > >> > > > >
>> > > > > > > > > > >> > > > > -Aaron
>> > > > > > > > > > >> > > > >
>> > > > > > > > > > >> > > > >
>> > > > > > > > > > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal
>> Mocny <
>> > > > > > > > > > >> mmocny@chromium.org>
>> > > > > > > > > > >> > > > > wrote:
>> > > > > > > > > > >> > > > >
>> > > > > > > > > > >> > > > > > Aaron,
>> > > > > > > > > > >> > > > > >
>> > > > > > > > > > >> > > > > > I haven't even begun looking at your
>> > > > implementation,
>> > > > > > but
>> > > > > > > > I'm
>> > > > > > > > > > >> just
>> > > > > > > > > > >> > > going
>> > > > > > > > > > >> > > > > to
>> > > > > > > > > > >> > > > > > say it: this is awesome!
>> > > > > > > > > > >> > > > > >
>> > > > > > > > > > >> > > > > > First question: When Capturing a DOM
>> element,
>> > > can
>> > > > > you
>> > > > > > > > > capture
>> > > > > > > > > > >> > 'body'
>> > > > > > > > > > >> > > to
>> > > > > > > > > > >> > > > > > grab it&children for a "full content
>> > > screenshot",
>> > > > or
>> > > > > > > does
>> > > > > > > > it
>> > > > > > > > > > >> have
>> > > > > > > > > > >> > to
>> > > > > > > > > > >> > > > be a
>> > > > > > > > > > >> > > > > > specific single element?
>> > > > > > > > > > >> > > > > >
>> > > > > > > > > > >> > > > > > -Michal
>> > > > > > > > > > >> > > > > >
>> > > > > > > > > > >> > > > > >
>> > > > > > > > > > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron
>> > > > Charbonneau <
>> > > > > > > > > > >> > > amcharbo@gmail.com
>> > > > > > > > > > >> > > > > > >wrote:
>> > > > > > > > > > >> > > > > >
>> > > > > > > > > > >> > > > > > > Greetings! My name is Aaron Charbonneau,
>> > > happy
>> > > > to
>> > > > > > be
>> > > > > > > a
>> > > > > > > > > new
>> > > > > > > > > > >> > member
>> > > > > > > > > > >> > > of
>> > > > > > > > > > >> > > > > the
>> > > > > > > > > > >> > > > > > > mailing list!
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > > I have been developing a sceenshot plugin
>> > for
>> > > > > > Cordova
>> > > > > > > to
>> > > > > > > > > > help
>> > > > > > > > > > >> > > > > facilitate
>> > > > > > > > > > >> > > > > > > automation testing and debugging of
>> Cordova
>> > > > apps,
>> > > > > > and
>> > > > > > > I
>> > > > > > > > > > would
>> > > > > > > > > > >> > love
>> > > > > > > > > > >> > > > some
>> > > > > > > > > > >> > > > > > > feedback on it. Currently Cordova
>> provides
>> > a
>> > > > > bunch
>> > > > > > of
>> > > > > > > > > > native
>> > > > > > > > > > >> > > > functions
>> > > > > > > > > > >> > > > > > > that allow you to do some cool stuff, but
>> > not
>> > > > much
>> > > > > > > > > > >> functionality
>> > > > > > > > > > >> > to
>> > > > > > > > > > >> > > > > test
>> > > > > > > > > > >> > > > > > > the apps that make use of them. Being
>> able
>> > to
>> > > > > take
>> > > > > > a
>> > > > > > > > > > capture
>> > > > > > > > > > >> of
>> > > > > > > > > > >> > > the
>> > > > > > > > > > >> > > > > > screen
>> > > > > > > > > > >> > > > > > > from within you app is a great way to
>> > automate
>> > > > > > testing
>> > > > > > > > or
>> > > > > > > > > > get
>> > > > > > > > > > >> > > > > additional
>> > > > > > > > > > >> > > > > > > information for debugging an issue.
>> Since
>> > > there
>> > > > > is
>> > > > > > no
>> > > > > > > > > > >> > > > > > > Javascript mechanism for taking screen
>> > > captures
>> > > > > the
>> > > > > > > > > solution
>> > > > > > > > > > >> > would
>> > > > > > > > > > >> > > > have
>> > > > > > > > > > >> > > > > > to
>> > > > > > > > > > >> > > > > > > be native, which fits nicely into the
>> "gap"
>> > > that
>> > > > > > > > > > >> Cordova/Phonegap
>> > > > > > > > > > >> > > > > > bridges.
>> > > > > > > > > > >> > > > > > > Any medium to large scale app can
>> benefit
>> > > > greatly
>> > > > > > > from
>> > > > > > > > > > >> > automation
>> > > > > > > > > > >> > > > > > testing
>> > > > > > > > > > >> > > > > > > and any app can benefit from an extra
>> > > debugging
>> > > > > > tool,
>> > > > > > > > and
>> > > > > > > > > > >> that is
>> > > > > > > > > > >> > > > what
>> > > > > > > > > > >> > > > > I
>> > > > > > > > > > >> > > > > > > hope this screenshot plugin can help
>> > achieve.
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > > Currently the plugin offers 2 functions:
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > > Capture():
>> > > > > > > > > > >> > > > > > > * Take a capture of the current view,
>> write
>> > > that
>> > > > > > > capture
>> > > > > > > > > to
>> > > > > > > > > > a
>> > > > > > > > > > >> > .png
>> > > > > > > > > > >> > > > file
>> > > > > > > > > > >> > > > > > > with the specified file name and sub
>> > directory
>> > > > of
>> > > > > > the
>> > > > > > > > > sdcard
>> > > > > > > > > > >> > > > (fallback
>> > > > > > > > > > >> > > > > to
>> > > > > > > > > > >> > > > > > > emulated sdcard in the case there isn't
>> an
>> > > > sdcard
>> > > > > > > > mounted)
>> > > > > > > > > > >> > > > > > > * Able to create a sub-screenshot with a
>> > > > specified
>> > > > > > > > > rectangle
>> > > > > > > > > > >> in
>> > > > > > > > > > >> > > order
>> > > > > > > > > > >> > > > > to
>> > > > > > > > > > >> > > > > > > block out ui elements that may be
>> variable,
>> > > and
>> > > > > also
>> > > > > > > > save
>> > > > > > > > > > >> space.
>> > > > > > > > > > >> > > > > > > * Can take captures of images/dom
>> elements
>> > > > > > (including
>> > > > > > > > > > canvas)
>> > > > > > > > > > >> > that
>> > > > > > > > > > >> > > > are
>> > > > > > > > > > >> > > > > > > lager than the actual screen size
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > > CaptureAndCompare():
>> > > > > > > > > > >> > > > > > > * All the functionality of Capture()
>> > > > > > > > > > >> > > > > > > * Perform a comparison between the
>> captured
>> > > > image
>> > > > > > and
>> > > > > > > a
>> > > > > > > > > > >> baseline
>> > > > > > > > > > >> > > > image
>> > > > > > > > > > >> > > > > > > located at the specified location in
>> either
>> > > the
>> > > > > > assets
>> > > > > > > > > > folder
>> > > > > > > > > > >> or
>> > > > > > > > > > >> > > the
>> > > > > > > > > > >> > > > > > > sdcard.
>> > > > > > > > > > >> > > > > > > * User can specify per color channel
>> > > tolerances
>> > > > as
>> > > > > > > well
>> > > > > > > > as
>> > > > > > > > > > >> total
>> > > > > > > > > > >> > > > pixel
>> > > > > > > > > > >> > > > > > > tolerances to avoid false positives for
>> the
>> > > > > > inevitable
>> > > > > > > > > > >> rendering
>> > > > > > > > > > >> > > > > > > differences across devices.
>> > > > > > > > > > >> > > > > > > * Optionally output a png file that
>> contains
>> > > the
>> > > > > > > > > differences
>> > > > > > > > > > >> > > between
>> > > > > > > > > > >> > > > > the
>> > > > > > > > > > >> > > > > > > actual and the baseline for
>> debugging/triage
>> > > > > > purposes,
>> > > > > > > > two
>> > > > > > > > > > >> modes:
>> > > > > > > > > > >> > > > > binary
>> > > > > > > > > > >> > > > > > > diff (all failing pixels appear as solid
>> > > white)
>> > > > or
>> > > > > > the
>> > > > > > > > > true
>> > > > > > > > > > >> > > > differences
>> > > > > > > > > > >> > > > > > > between pixels.
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > > If you can spare some time, I would love
>> it
>> > if
>> > > > you
>> > > > > > > could
>> > > > > > > > > > take
>> > > > > > > > > > >> a
>> > > > > > > > > > >> > > look
>> > > > > > > > > > >> > > > at
>> > > > > > > > > > >> > > > > > the
>> > > > > > > > > > >> > > > > > > api and parameters I have defined to make
>> > sure
>> > > > > they
>> > > > > > > > adhere
>> > > > > > > > > > to
>> > > > > > > > > > >> > > Cordova
>> > > > > > > > > > >> > > > > > > plugin best practices. The most crucial
>> > part
>> > > > > would
>> > > > > > be
>> > > > > > > > in
>> > > > > > > > > > the
>> > > > > > > > > > >> > > plugin
>> > > > > > > > > > >> > > > > > itself
>> > > > > > > > > > >> > > > > > > at ScreenCapture.java:
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > >
>> > > > > > > > > > >> > > > >
>> > > > > > > > > > >> > > >
>> > > > > > > > > > >> > >
>> > > > > > > > > > >> >
>> > > > > > > > > > >>
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
>> > > > > > > > > > >> > > > > > > and captureAndCompare() are the two
>> exposed
>> > > > > > functions)
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > > I'm also interested to know you thoughts
>> on
>> > > it's
>> > > > > > > usage.
>> > > > > > > > I
>> > > > > > > > > > >> > started
>> > > > > > > > > > >> > > > with
>> > > > > > > > > > >> > > > > > the
>> > > > > > > > > > >> > > > > > > mobile-spec testing framework and put
>> some
>> > > quick
>> > > > > > > > rendering
>> > > > > > > > > > >> tests
>> > > > > > > > > > >> > > into
>> > > > > > > > > > >> > > > > the
>> > > > > > > > > > >> > > > > > > Autotest section as a viable use case.
>> In
>> > > order
>> > > > > to
>> > > > > > > get
>> > > > > > > > > the
>> > > > > > > > > > >> > WebView
>> > > > > > > > > > >> > > > > into
>> > > > > > > > > > >> > > > > > > the state I wanted to capture, I had to
>> > > > implement
>> > > > > a
>> > > > > > > wait
>> > > > > > > > > > >> timer to
>> > > > > > > > > > >> > > > allow
>> > > > > > > > > > >> > > > > > the
>> > > > > > > > > > >> > > > > > > view to update before taking the capture.
>> > > Once
>> > > > > the
>> > > > > > > wait
>> > > > > > > > > > >> function
>> > > > > > > > > > >> > > > > > > completes, the capture can be taken and
>> > > > everything
>> > > > > > > from
>> > > > > > > > > > there
>> > > > > > > > > > >> is
>> > > > > > > > > > >> > > > > callback
>> > > > > > > > > > >> > > > > > > based. I use the Jasmine
>> waitsFor()/runs()
>> > > > blocks
>> > > > > > to
>> > > > > > > > make
>> > > > > > > > > > the
>> > > > > > > > > > >> > > tests
>> > > > > > > > > > >> > > > > run
>> > > > > > > > > > >> > > > > > > synchronously. For some validation
>> testing
>> > I
>> > > > > made a
>> > > > > > > > > compare
>> > > > > > > > > > >> > > function
>> > > > > > > > > > >> > > > > in
>> > > > > > > > > > >> > > > > > > Javascript to run against the native
>> > compare.
>> > > > > Turns
>> > > > > > > out
>> > > > > > > > > > Java
>> > > > > > > > > > >> > runs
>> > > > > > > > > > >> > > > alot
>> > > > > > > > > > >> > > > > > > faster than Javascript (surprise!) thus
>> the
>> > > > native
>> > > > > > > > compare
>> > > > > > > > > > is
>> > > > > > > > > > >> > about
>> > > > > > > > > > >> > > > > 2-5x
>> > > > > > > > > > >> > > > > > > faster. All in all the process is fairly
>> > > quick,
>> > > > > the
>> > > > > > > > full
>> > > > > > > > > > >> > > > > > captureAndCompare
>> > > > > > > > > > >> > > > > > > with 3 file io's takes about 233ms on a
>> > Nexus
>> > > > One,
>> > > > > > and
>> > > > > > > > > > ~100ms
>> > > > > > > > > > >> on
>> > > > > > > > > > >> > a
>> > > > > > > > > > >> > > > > Nexus
>> > > > > > > > > > >> > > > > > 10
>> > > > > > > > > > >> > > > > > > (256x256 image size).
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > > Anyways here is the usage:
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > >
>> > > > > > > > > > >> > > > >
>> > > > > > > > > > >> > > >
>> > > > > > > > > > >> > >
>> > > > > > > > > > >> >
>> > > > > > > > > > >>
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > > And here's the JS wrapper for the plugin
>> > > calls:
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > >
>> > > > > > > > > > >> > > > >
>> > > > > > > > > > >> > > >
>> > > > > > > > > > >> > >
>> > > > > > > > > > >> >
>> > > > > > > > > > >>
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > > Thanks for your time, and I look forward
>> to
>> > > ANY
>> > > > > > > feedback
>> > > > > > > > > > >> positive
>> > > > > > > > > > >> > > or
>> > > > > > > > > > >> > > > > > > negative.
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > > > Thanks,
>> > > > > > > > > > >> > > > > > > Aaron
>> > > > > > > > > > >> > > > > > >
>> > > > > > > > > > >> > > > > >
>> > > > > > > > > > >> > > > >
>> > > > > > > > > > >> > > >
>> > > > > > > > > > >> > >
>> > > > > > > > > > >> >
>> > > > > > > > > > >>
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: Cordova Screen Capture Plugin for Android
Posted by Aaron Charbonneau <am...@gmail.com>.
Cool thanks for explaining that, alot of the information for Cordova plugin
dev is a little out-dated, makes it hard sometimes to 'do the right thing.'
Ok I've made all the changes:
https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
I'll probably go forward with this approach once I've written some more
tests to make sure there's no issues, and test out the additional
functionality for throughput captures.
Thanks again the help and feedback, was a very good learning experience.
Please let me know if you see any other issues!
Thanks,
Aaron
On Mon, Mar 18, 2013 at 1:44 PM, Andrew Grieve <ag...@chromium.org> wrote:
> On Mon, Mar 18, 2013 at 4:38 PM, Aaron Charbonneau <amcharbo@gmail.com
> >wrote:
>
> > "You could avoid the blocking queue if you dispatch the background
> thread's
> > runnable at the end of the UI thread's block."
> >
> > -Great idea, done.
> >
> >
> > I traced through to see what sending a NO_RESULT PluginResult with
> > keepCallback==true does, and it looks like a no-op
> > (NativeToJsMessageQueue.java, addPluginResult()):
> >
> > boolean noResult = result.getStatus() ==
> > PluginResult.Status.NO_RESULT.ordinal();
> > boolean keepCallback = result.getKeepCallback();
> > if (noResult && keepCallback) {
> > return;
> > }
> >
> > I'm sure I'm missing something here, is that really a no-op?
>
> Yep, that's a no-op and exists only to support the previous style of
> Android plugins, where the execute method *had* to return a PluginResult.
>
>
>
> > Did you
> > mean for me to send a PluginResult.Status.OK with keepCallBack=true?
> >
> That's exactly what I meant :)
>
>
> >
> >
> > Thanks,
> > Aaron
> >
> >
> >
> >
> > On Sat, Mar 16, 2013 at 6:15 AM, Andrew Grieve <ag...@chromium.org>
> > wrote:
> >
> > > You could avoid the blocking queue if you dispatch the background
> > thread's
> > > runnable at the end of the UI thread's block.
> > >
> > > I see you send a success callback as soon as the image is captured, but
> > > before it is saved. This would improve throughput, but it might be even
> > > more useful if you set the keepCallback flag on it and then send
> another
> > > success response once you have the filename / comparison results. It's
> > the
> > > filename that the app is probably interested in.
> > >
> > >
> > > On Fri, Mar 15, 2013 at 9:41 PM, Aaron Charbonneau <amcharbo@gmail.com
> > > >wrote:
> > >
> > > > Ah thanks for clarifying that.
> > > >
> > > > I'm not sure how I can alleviate the ui thread any more than I am
> > > > currently. It is already doing the bare minimum amount of work on
> the
> > > > there. Right now the only work done on the ui thread is
> > > > view.capturePicture() which is the recommended thread for doing that
> > work
> > > > (otherwise you get a warning):
> > > >
> > > >
> > > > mUIThreadDone = false;
> > > > cordova.getActivity().runOnUiThread(new Runnable() {
> > > > public void run() {
> > > > CordovaWebView uiThreadView = webView;
> > > > picture = uiThreadView.capturePicture();
> > > > mUIThreadDone = true;
> > > > }
> > > > });
> > > > //rest of the work is done on core thread
> > > > while(!mUIThreadDone) {}
> > > > //write the picture to a file
> > > > ...
> > > >
> > > > I could do the file io and compares on a different thread than the
> > > > core one, but for the major use case, the core thread needs to wait
> no
> > > > matter what to ensure that the file is written and the comparison is
> > > > complete, so I wouldn't expect much gain there.
> > > >
> > > >
> > > > However, I've been experimenting with a more threaded version that
> > > > does the file io and comparison on a background thread as you
> > > > suggested. If you have time please check it out. While the code is
> > > > not as clean, I think it addresses the issues you raised Andrew. For
> > > > the major use case, performance is still the same as expected, but if
> > > > we want pure throughput on captures there's a big gain from about 3
> > > > captures per second (cps) to 10 cps on a Nexus 10.
> > > >
> > > >
> > > > Here's the new plugin, with moar threading!
> > > >
> > > >
> > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
> > > >
> > > >
> > > > Once again, thanks for all your feedback on this, I haven't had much
> > > > experience with threading in java before now.
> > > >
> > > >
> > > > -Aaron
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Thu, Mar 14, 2013 at 8:10 AM, Andrew Grieve <agrieve@chromium.org
> >
> > > > wrote:
> > > >
> > > > > My suggestion was just to prevent the UI thread from locking up so
> > that
> > > > you
> > > > > app doesn't appear to be frozen when the plugin is doing its thing.
> > > > >
> > > > >
> > > > > On Wed, Mar 13, 2013 at 5:03 PM, Aaron Charbonneau <
> > amcharbo@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Yah that's how I have it working now :) No problem with that
> > approach
> > > > for
> > > > > > working with Jasmine.
> > > > > >
> > > > > > I made reference to running them asynchronously to respond to
> > Andrew
> > > > > about
> > > > > > returning immediately after the bits are captured, which in the
> > case
> > > of
> > > > > > using that with Jasmine would only see a benefit IF we could run
> > > > multiple
> > > > > > 'it' blocks at the same time.
> > > > > >
> > > > > > I think however Andrew was thinking of a different use case where
> > we
> > > > > aren't
> > > > > > limited by a framework that requires tests to run serially, and
> > > results
> > > > > can
> > > > > > just come back as they are ready, and be evaluated all together
> at
> > > the
> > > > > end,
> > > > > > or not at all if you just want the captures and no evaluation.
> > > > > >
> > > > > > -Aaron
> > > > > >
> > > > > >
> > > > > > On Wed, Mar 13, 2013 at 1:53 PM, Braden Shepherdson <
> > > > braden@chromium.org
> > > > > > >wrote:
> > > > > >
> > > > > > > As far as I know, you're correct: there's no way to move on the
> > > next
> > > > > it()
> > > > > > > block before the first one is done.
> > > > > > >
> > > > > > > What's the problem with just making the tests wait? Why does
> the
> > > next
> > > > > > it()
> > > > > > > block need to start early? You can make your call to Java, use
> > > > waitFor
> > > > > to
> > > > > > > wait until the callback with the data is called, run your
> > > > expectations,
> > > > > > and
> > > > > > > then move on to the next it() block. If the next it() block is
> > > using
> > > > > the
> > > > > > > result of this one, then they should be one it() block.
> > > > > > >
> > > > > > > Braden
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Mar 13, 2013 at 4:46 PM, Aaron Charbonneau <
> > > > amcharbo@gmail.com
> > > > > > > >wrote:
> > > > > > >
> > > > > > > > Hi Braden,
> > > > > > > > Sorry maybe I am overlooking something. The way I understand
> > it
> > > is
> > > > > > that
> > > > > > > a
> > > > > > > > previous 'it' block needs to complete before the next one can
> > > run.
> > > > > So
> > > > > > I
> > > > > > > > have various tests setup like this:
> > > > > > > >
> > > > > > > > it("renders something", function() {
> > > > > > > > //draw something
> > > > > > > > ....
> > > > > > > > captureAndCompare(); //call the native function
> > > > > > > > waitsFor(function() {
> > > > > > > > return captureComplete; //set by the native function callback
> > > > > > > > }, "capture never completed", 10000);
> > > > > > > > runs(function() {
> > > > > > > > expect(captureResult).toBe(0); //expects difference to be 0
> > > > > > > > });
> > > > > > > > });
> > > > > > > > //next 'it' test
> > > > > > > > ...
> > > > > > > >
> > > > > > > > The expect block needs to wait until there is a result
> returned
> > > > from
> > > > > > the
> > > > > > > > native captureAndCompare call. Therefore we use a
> > waitsFor/runs
> > > > > block
> > > > > > > > there.
> > > > > > > >
> > > > > > > > But say we wanted to allow Javascript to run immediately
> after
> > > > > calling
> > > > > > > that
> > > > > > > > native function. The only thing you need to wait for is that
> > > Java
> > > > > > > grabbed
> > > > > > > > the bits, and the rest of the processing will be done 'at
> some
> > > > point
> > > > > in
> > > > > > > the
> > > > > > > > future.' So as Andrew suggested you would return as soon as
> > the
> > > > bits
> > > > > > are
> > > > > > > > captured, and allow JS to run again. The problem is that our
> > > > expect
> > > > > > > block
> > > > > > > > needs the results, so that has to wait, the only other thing
> to
> > > do
> > > > > > would
> > > > > > > be
> > > > > > > > to run the next 'it' block.
> > > > > > > >
> > > > > > > > This is the part I didn't think was possible, can the next
> 'it'
> > > > block
> > > > > > be
> > > > > > > > run while the previous 'it' is still waiting to execute the
> > > > 'expect'
> > > > > > > block?
> > > > > > > > Or is there another approach all together that would allow
> > this?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Aaron
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <
> > > > > > braden@chromium.org
> > > > > > > > >wrote:
> > > > > > > >
> > > > > > > > > Jasmine does support asynchronous tests, and waiting an
> > > arbitrary
> > > > > > > period
> > > > > > > > or
> > > > > > > > > until some condition is true. Why does that not work for
> > these
> > > > > tests?
> > > > > > > > >
> > > > > > > > > Braden
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <
> > > > > > amcharbo@gmail.com
> > > > > > > > > >wrote:
> > > > > > > > >
> > > > > > > > > > Quick follow up question for the return values stuff:
> > > > > > > > > > Does it make sense to have my functions called from
> execute
> > > > > > > (capture()
> > > > > > > > > and
> > > > > > > > > > captureAndCompare()) return an error string if one is
> > > > encountered
> > > > > > and
> > > > > > > > > pass
> > > > > > > > > > that back in callbackContext.error, then have execute
> > return
> > > > > true?
> > > > > > > > > >
> > > > > > > > > > Or is there a preferred way to get errors from sub
> > functions
> > > > back
> > > > > > > into
> > > > > > > > > > Javascript?
> > > > > > > > > >
> > > > > > > > > > Thanks!
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <
> > > > > > > > amcharbo@gmail.com
> > > > > > > > > > >wrote:
> > > > > > > > > >
> > > > > > > > > > > Thanks Andrew these are great suggestions!
> > > > > > > > > > >
> > > > > > > > > > > About not needing the busy-wait in getScreenBits:
> > > > > > > > > > > I'm not a fan of busy-waiting either.
> > > > > > > > > > > Perhaps it's a side effect of Jasmine requiring tests
> to
> > be
> > > > run
> > > > > > and
> > > > > > > > > > > evaluated synchronously that I decided to use a
> busy-wait
> > > in
> > > > > > > > > > getScreenBits.
> > > > > > > > > > > If the Runnable calls CallbackContext.success/error
> once
> > > the
> > > > > > bits
> > > > > > > > are
> > > > > > > > > > > captured (but not written to file) on the UIThread, the
> > > > > > Javascript
> > > > > > > > side
> > > > > > > > > > > will proceed to run before it has an image url/compare
> > > > result,
> > > > > > most
> > > > > > > > > > likely
> > > > > > > > > > > causing that test to fail. As far as I know Jasmine
> > > doesn't
> > > > > > allow
> > > > > > > > for
> > > > > > > > > > > kicking off a bunch of tests and then gather all the
> > > results
> > > > at
> > > > > > the
> > > > > > > > end
> > > > > > > > > > and
> > > > > > > > > > > output pass/fails (each 'expect' function needs to be
> > > within
> > > > an
> > > > > > > 'it'
> > > > > > > > > > block,
> > > > > > > > > > > essentially that means one test must be evaluated
> before
> > > the
> > > > > next
> > > > > > > can
> > > > > > > > > > > begin).
> > > > > > > > > > >
> > > > > > > > > > > I can see the usefulness of allowing the capture, file
> > io,
> > > > and
> > > > > > > > > comparison
> > > > > > > > > > > to be able to run asynchronously from the Javascript
> > > though.
> > > > In
> > > > > > the
> > > > > > > > > case
> > > > > > > > > > > where you have your own test framework that CAN kick
> off
> > a
> > > > > bunch
> > > > > > of
> > > > > > > > > tests
> > > > > > > > > > > and just gather the results as they come in and do a
> > batch
> > > > > > > evaluation
> > > > > > > > > at
> > > > > > > > > > > the end, or in the case where you don't want to do any
> > > > > evaluation
> > > > > > > in
> > > > > > > > > your
> > > > > > > > > > > app and just want to get captures as fast as you can,
> > > > > > asynchronous
> > > > > > > > > could
> > > > > > > > > > > increase performance there.
> > > > > > > > > > >
> > > > > > > > > > > I could try to implement it this way too, if you think
> > > > that's a
> > > > > > > worth
> > > > > > > > > > > while use case, maybe just provide an 'asynchronous'
> flag
> > > in
> > > > > > > > > > CaptureOptions?
> > > > > > > > > > >
> > > > > > > > > > > For doing the fileIO/compare using
> cordova.getThreadPool,
> > > is
> > > > > > there
> > > > > > > a
> > > > > > > > > > > benefit to doing that if I still plan having the
> > operation
> > > > > block
> > > > > > > the
> > > > > > > > > > > Javascript? For asynchronous mode I see why that makes
> > > > sense.
> > > > > > > > > > >
> > > > > > > > > > > For the return values, I'll make those changes right
> > away!
> > > > > > Thanks
> > > > > > > > > again
> > > > > > > > > > > for providing your insight, hopefully I understood it
> > > > > correctly.
> > > > > > > > > > >
> > > > > > > > > > > -Aaron
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <
> > > > > > > agrieve@chromium.org
> > > > > > > > > > >wrote:
> > > > > > > > > > >
> > > > > > > > > > >> Had a glance at your code. Things you may want to
> > address:
> > > > > > > > > > >>
> > > > > > > > > > >> while(!mUIThreadDone) {}
> > > > > > > > > > >>
> > > > > > > > > > >> You shouldn't need to busy-wait. Pass the
> > CallbackContext
> > > to
> > > > > the
> > > > > > > > > > Runnable,
> > > > > > > > > > >> and call success/error on it whenever you're done. No
> > need
> > > > to
> > > > > > have
> > > > > > > > the
> > > > > > > > > > web
> > > > > > > > > > >> thread wait for it. Even better would be to use a
> second
> > > > > > Runnable
> > > > > > > to
> > > > > > > > > do
> > > > > > > > > > >> the
> > > > > > > > > > >> image compare / file IO on a background thread via
> > > > > > > > > > cordova.getThreadPool()
> > > > > > > > > > >>
> > > > > > > > > > >> The return value to execute() should only be false if
> an
> > > > > invalid
> > > > > > > > > command
> > > > > > > > > > >> was passed. It's not meant to indicate the success of
> > the
> > > > > > > operation.
> > > > > > > > > You
> > > > > > > > > > >> should be returning true for it.
> > > > > > > > > > >>
> > > > > > > > > > >> Instead of return "success"/"", it might simplify
> > things a
> > > > bit
> > > > > > to
> > > > > > > > > > return a
> > > > > > > > > > >> boolean of true/false
> > > > > > > > > > >>
> > > > > > > > > > >>
> > > > > > > > > > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <
> > > > > > > > lorin.beer.dev@gmail.com
> > > > > > > > > > >> >wrote:
> > > > > > > > > > >>
> > > > > > > > > > >> > all the same, very impressive and faster then I
> > > expected.
> > > > > > > > > > >> >
> > > > > > > > > > >> > - Lorin
> > > > > > > > > > >> >
> > > > > > > > > > >> >
> > > > > > > > > > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <
> > > > > > > > > > amcharbo@gmail.com
> > > > > > > > > > >> > >wrote:
> > > > > > > > > > >> >
> > > > > > > > > > >> > > Thanks Lorin.
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > I did a quick test for throughput on an animating
> > > > canvas.
> > > > > I
> > > > > > > was
> > > > > > > > > > able
> > > > > > > > > > >> to
> > > > > > > > > > >> > > grab 30 frames within a 5 seconds which equates to
> > > > around
> > > > > > > 166ms
> > > > > > > > > per
> > > > > > > > > > >> > capture
> > > > > > > > > > >> > > on a Galaxy Nexus. Unfortunately not fast enough
> to
> > > > > > automate
> > > > > > > > > > >> something
> > > > > > > > > > >> > > like video. As with any kind of capturing system,
> > > often
> > > > > the
> > > > > > > > > > overhead
> > > > > > > > > > >> of
> > > > > > > > > > >> > > copying bits around is enough to slow down the
> > > operation
> > > > > of
> > > > > > > the
> > > > > > > > > app.
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > -Aaron
> > > > > > > > > > >> > >
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
> > > > > > > > > > >> lorin.beer.dev@gmail.com
> > > > > > > > > > >> > > >wrote:
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > > Hey Aaron,
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > > very cool stuff, looking forward to checking it
> > out!
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > > Question: any performance hit on the app by
> using
> > > > > Capture?
> > > > > > > > > What's
> > > > > > > > > > >> the
> > > > > > > > > > >> > > > expected throughput on images taken in this way
> > > > (given a
> > > > > > > > > > particular
> > > > > > > > > > >> > > device,
> > > > > > > > > > >> > > > say Galaxy Nexus)?
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > > - Lorin
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron
> > Charbonneau
> > > <
> > > > > > > > > > >> > amcharbo@gmail.com
> > > > > > > > > > >> > > > >wrote:
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > > > Glad you like it :)
> > > > > > > > > > >> > > > > Yes in fact the actual capture makes use of
> > > > > > > > > > View.capturePicture()
> > > > > > > > > > >> > which
> > > > > > > > > > >> > > > > actually grabs the entire document, then that
> > can
> > > be
> > > > > > > clipped
> > > > > > > > > > down
> > > > > > > > > > >> to
> > > > > > > > > > >> > > the
> > > > > > > > > > >> > > > > size/location of a specific element.
> > > > > > > > > > >> > > > >
> > > > > > > > > > >> > > > > -Aaron
> > > > > > > > > > >> > > > >
> > > > > > > > > > >> > > > >
> > > > > > > > > > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal
> Mocny <
> > > > > > > > > > >> mmocny@chromium.org>
> > > > > > > > > > >> > > > > wrote:
> > > > > > > > > > >> > > > >
> > > > > > > > > > >> > > > > > Aaron,
> > > > > > > > > > >> > > > > >
> > > > > > > > > > >> > > > > > I haven't even begun looking at your
> > > > implementation,
> > > > > > but
> > > > > > > > I'm
> > > > > > > > > > >> just
> > > > > > > > > > >> > > going
> > > > > > > > > > >> > > > > to
> > > > > > > > > > >> > > > > > say it: this is awesome!
> > > > > > > > > > >> > > > > >
> > > > > > > > > > >> > > > > > First question: When Capturing a DOM
> element,
> > > can
> > > > > you
> > > > > > > > > capture
> > > > > > > > > > >> > 'body'
> > > > > > > > > > >> > > to
> > > > > > > > > > >> > > > > > grab it&children for a "full content
> > > screenshot",
> > > > or
> > > > > > > does
> > > > > > > > it
> > > > > > > > > > >> have
> > > > > > > > > > >> > to
> > > > > > > > > > >> > > > be a
> > > > > > > > > > >> > > > > > specific single element?
> > > > > > > > > > >> > > > > >
> > > > > > > > > > >> > > > > > -Michal
> > > > > > > > > > >> > > > > >
> > > > > > > > > > >> > > > > >
> > > > > > > > > > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron
> > > > Charbonneau <
> > > > > > > > > > >> > > amcharbo@gmail.com
> > > > > > > > > > >> > > > > > >wrote:
> > > > > > > > > > >> > > > > >
> > > > > > > > > > >> > > > > > > Greetings! My name is Aaron Charbonneau,
> > > happy
> > > > to
> > > > > > be
> > > > > > > a
> > > > > > > > > new
> > > > > > > > > > >> > member
> > > > > > > > > > >> > > of
> > > > > > > > > > >> > > > > the
> > > > > > > > > > >> > > > > > > mailing list!
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > I have been developing a sceenshot plugin
> > for
> > > > > > Cordova
> > > > > > > to
> > > > > > > > > > help
> > > > > > > > > > >> > > > > facilitate
> > > > > > > > > > >> > > > > > > automation testing and debugging of
> Cordova
> > > > apps,
> > > > > > and
> > > > > > > I
> > > > > > > > > > would
> > > > > > > > > > >> > love
> > > > > > > > > > >> > > > some
> > > > > > > > > > >> > > > > > > feedback on it. Currently Cordova
> provides
> > a
> > > > > bunch
> > > > > > of
> > > > > > > > > > native
> > > > > > > > > > >> > > > functions
> > > > > > > > > > >> > > > > > > that allow you to do some cool stuff, but
> > not
> > > > much
> > > > > > > > > > >> functionality
> > > > > > > > > > >> > to
> > > > > > > > > > >> > > > > test
> > > > > > > > > > >> > > > > > > the apps that make use of them. Being
> able
> > to
> > > > > take
> > > > > > a
> > > > > > > > > > capture
> > > > > > > > > > >> of
> > > > > > > > > > >> > > the
> > > > > > > > > > >> > > > > > screen
> > > > > > > > > > >> > > > > > > from within you app is a great way to
> > automate
> > > > > > testing
> > > > > > > > or
> > > > > > > > > > get
> > > > > > > > > > >> > > > > additional
> > > > > > > > > > >> > > > > > > information for debugging an issue. Since
> > > there
> > > > > is
> > > > > > no
> > > > > > > > > > >> > > > > > > Javascript mechanism for taking screen
> > > captures
> > > > > the
> > > > > > > > > solution
> > > > > > > > > > >> > would
> > > > > > > > > > >> > > > have
> > > > > > > > > > >> > > > > > to
> > > > > > > > > > >> > > > > > > be native, which fits nicely into the
> "gap"
> > > that
> > > > > > > > > > >> Cordova/Phonegap
> > > > > > > > > > >> > > > > > bridges.
> > > > > > > > > > >> > > > > > > Any medium to large scale app can benefit
> > > > greatly
> > > > > > > from
> > > > > > > > > > >> > automation
> > > > > > > > > > >> > > > > > testing
> > > > > > > > > > >> > > > > > > and any app can benefit from an extra
> > > debugging
> > > > > > tool,
> > > > > > > > and
> > > > > > > > > > >> that is
> > > > > > > > > > >> > > > what
> > > > > > > > > > >> > > > > I
> > > > > > > > > > >> > > > > > > hope this screenshot plugin can help
> > achieve.
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > Currently the plugin offers 2 functions:
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > Capture():
> > > > > > > > > > >> > > > > > > * Take a capture of the current view,
> write
> > > that
> > > > > > > capture
> > > > > > > > > to
> > > > > > > > > > a
> > > > > > > > > > >> > .png
> > > > > > > > > > >> > > > file
> > > > > > > > > > >> > > > > > > with the specified file name and sub
> > directory
> > > > of
> > > > > > the
> > > > > > > > > sdcard
> > > > > > > > > > >> > > > (fallback
> > > > > > > > > > >> > > > > to
> > > > > > > > > > >> > > > > > > emulated sdcard in the case there isn't an
> > > > sdcard
> > > > > > > > mounted)
> > > > > > > > > > >> > > > > > > * Able to create a sub-screenshot with a
> > > > specified
> > > > > > > > > rectangle
> > > > > > > > > > >> in
> > > > > > > > > > >> > > order
> > > > > > > > > > >> > > > > to
> > > > > > > > > > >> > > > > > > block out ui elements that may be
> variable,
> > > and
> > > > > also
> > > > > > > > save
> > > > > > > > > > >> space.
> > > > > > > > > > >> > > > > > > * Can take captures of images/dom elements
> > > > > > (including
> > > > > > > > > > canvas)
> > > > > > > > > > >> > that
> > > > > > > > > > >> > > > are
> > > > > > > > > > >> > > > > > > lager than the actual screen size
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > CaptureAndCompare():
> > > > > > > > > > >> > > > > > > * All the functionality of Capture()
> > > > > > > > > > >> > > > > > > * Perform a comparison between the
> captured
> > > > image
> > > > > > and
> > > > > > > a
> > > > > > > > > > >> baseline
> > > > > > > > > > >> > > > image
> > > > > > > > > > >> > > > > > > located at the specified location in
> either
> > > the
> > > > > > assets
> > > > > > > > > > folder
> > > > > > > > > > >> or
> > > > > > > > > > >> > > the
> > > > > > > > > > >> > > > > > > sdcard.
> > > > > > > > > > >> > > > > > > * User can specify per color channel
> > > tolerances
> > > > as
> > > > > > > well
> > > > > > > > as
> > > > > > > > > > >> total
> > > > > > > > > > >> > > > pixel
> > > > > > > > > > >> > > > > > > tolerances to avoid false positives for
> the
> > > > > > inevitable
> > > > > > > > > > >> rendering
> > > > > > > > > > >> > > > > > > differences across devices.
> > > > > > > > > > >> > > > > > > * Optionally output a png file that
> contains
> > > the
> > > > > > > > > differences
> > > > > > > > > > >> > > between
> > > > > > > > > > >> > > > > the
> > > > > > > > > > >> > > > > > > actual and the baseline for
> debugging/triage
> > > > > > purposes,
> > > > > > > > two
> > > > > > > > > > >> modes:
> > > > > > > > > > >> > > > > binary
> > > > > > > > > > >> > > > > > > diff (all failing pixels appear as solid
> > > white)
> > > > or
> > > > > > the
> > > > > > > > > true
> > > > > > > > > > >> > > > differences
> > > > > > > > > > >> > > > > > > between pixels.
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > If you can spare some time, I would love
> it
> > if
> > > > you
> > > > > > > could
> > > > > > > > > > take
> > > > > > > > > > >> a
> > > > > > > > > > >> > > look
> > > > > > > > > > >> > > > at
> > > > > > > > > > >> > > > > > the
> > > > > > > > > > >> > > > > > > api and parameters I have defined to make
> > sure
> > > > > they
> > > > > > > > adhere
> > > > > > > > > > to
> > > > > > > > > > >> > > Cordova
> > > > > > > > > > >> > > > > > > plugin best practices. The most crucial
> > part
> > > > > would
> > > > > > be
> > > > > > > > in
> > > > > > > > > > the
> > > > > > > > > > >> > > plugin
> > > > > > > > > > >> > > > > > itself
> > > > > > > > > > >> > > > > > > at ScreenCapture.java:
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > >
> > > > > > > > > > >> > > > >
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > >
> > > > > > > > > > >> >
> > > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > > > > > > > > >> > > > > > > and captureAndCompare() are the two
> exposed
> > > > > > functions)
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > I'm also interested to know you thoughts
> on
> > > it's
> > > > > > > usage.
> > > > > > > > I
> > > > > > > > > > >> > started
> > > > > > > > > > >> > > > with
> > > > > > > > > > >> > > > > > the
> > > > > > > > > > >> > > > > > > mobile-spec testing framework and put some
> > > quick
> > > > > > > > rendering
> > > > > > > > > > >> tests
> > > > > > > > > > >> > > into
> > > > > > > > > > >> > > > > the
> > > > > > > > > > >> > > > > > > Autotest section as a viable use case. In
> > > order
> > > > > to
> > > > > > > get
> > > > > > > > > the
> > > > > > > > > > >> > WebView
> > > > > > > > > > >> > > > > into
> > > > > > > > > > >> > > > > > > the state I wanted to capture, I had to
> > > > implement
> > > > > a
> > > > > > > wait
> > > > > > > > > > >> timer to
> > > > > > > > > > >> > > > allow
> > > > > > > > > > >> > > > > > the
> > > > > > > > > > >> > > > > > > view to update before taking the capture.
> > > Once
> > > > > the
> > > > > > > wait
> > > > > > > > > > >> function
> > > > > > > > > > >> > > > > > > completes, the capture can be taken and
> > > > everything
> > > > > > > from
> > > > > > > > > > there
> > > > > > > > > > >> is
> > > > > > > > > > >> > > > > callback
> > > > > > > > > > >> > > > > > > based. I use the Jasmine
> waitsFor()/runs()
> > > > blocks
> > > > > > to
> > > > > > > > make
> > > > > > > > > > the
> > > > > > > > > > >> > > tests
> > > > > > > > > > >> > > > > run
> > > > > > > > > > >> > > > > > > synchronously. For some validation
> testing
> > I
> > > > > made a
> > > > > > > > > compare
> > > > > > > > > > >> > > function
> > > > > > > > > > >> > > > > in
> > > > > > > > > > >> > > > > > > Javascript to run against the native
> > compare.
> > > > > Turns
> > > > > > > out
> > > > > > > > > > Java
> > > > > > > > > > >> > runs
> > > > > > > > > > >> > > > alot
> > > > > > > > > > >> > > > > > > faster than Javascript (surprise!) thus
> the
> > > > native
> > > > > > > > compare
> > > > > > > > > > is
> > > > > > > > > > >> > about
> > > > > > > > > > >> > > > > 2-5x
> > > > > > > > > > >> > > > > > > faster. All in all the process is fairly
> > > quick,
> > > > > the
> > > > > > > > full
> > > > > > > > > > >> > > > > > captureAndCompare
> > > > > > > > > > >> > > > > > > with 3 file io's takes about 233ms on a
> > Nexus
> > > > One,
> > > > > > and
> > > > > > > > > > ~100ms
> > > > > > > > > > >> on
> > > > > > > > > > >> > a
> > > > > > > > > > >> > > > > Nexus
> > > > > > > > > > >> > > > > > 10
> > > > > > > > > > >> > > > > > > (256x256 image size).
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > Anyways here is the usage:
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > >
> > > > > > > > > > >> > > > >
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > >
> > > > > > > > > > >> >
> > > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > And here's the JS wrapper for the plugin
> > > calls:
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > >
> > > > > > > > > > >> > > > >
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > >
> > > > > > > > > > >> >
> > > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > Thanks for your time, and I look forward
> to
> > > ANY
> > > > > > > feedback
> > > > > > > > > > >> positive
> > > > > > > > > > >> > > or
> > > > > > > > > > >> > > > > > > negative.
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > Thanks,
> > > > > > > > > > >> > > > > > > Aaron
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > >
> > > > > > > > > > >> > > > >
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > >
> > > > > > > > > > >> >
> > > > > > > > > > >>
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Andrew Grieve <ag...@chromium.org>.
On Mon, Mar 18, 2013 at 4:38 PM, Aaron Charbonneau <am...@gmail.com>wrote:
> "You could avoid the blocking queue if you dispatch the background thread's
> runnable at the end of the UI thread's block."
>
> -Great idea, done.
>
>
> I traced through to see what sending a NO_RESULT PluginResult with
> keepCallback==true does, and it looks like a no-op
> (NativeToJsMessageQueue.java, addPluginResult()):
>
> boolean noResult = result.getStatus() ==
> PluginResult.Status.NO_RESULT.ordinal();
> boolean keepCallback = result.getKeepCallback();
> if (noResult && keepCallback) {
> return;
> }
>
> I'm sure I'm missing something here, is that really a no-op?
Yep, that's a no-op and exists only to support the previous style of
Android plugins, where the execute method *had* to return a PluginResult.
> Did you
> mean for me to send a PluginResult.Status.OK with keepCallBack=true?
>
That's exactly what I meant :)
>
>
> Thanks,
> Aaron
>
>
>
>
> On Sat, Mar 16, 2013 at 6:15 AM, Andrew Grieve <ag...@chromium.org>
> wrote:
>
> > You could avoid the blocking queue if you dispatch the background
> thread's
> > runnable at the end of the UI thread's block.
> >
> > I see you send a success callback as soon as the image is captured, but
> > before it is saved. This would improve throughput, but it might be even
> > more useful if you set the keepCallback flag on it and then send another
> > success response once you have the filename / comparison results. It's
> the
> > filename that the app is probably interested in.
> >
> >
> > On Fri, Mar 15, 2013 at 9:41 PM, Aaron Charbonneau <amcharbo@gmail.com
> > >wrote:
> >
> > > Ah thanks for clarifying that.
> > >
> > > I'm not sure how I can alleviate the ui thread any more than I am
> > > currently. It is already doing the bare minimum amount of work on the
> > > there. Right now the only work done on the ui thread is
> > > view.capturePicture() which is the recommended thread for doing that
> work
> > > (otherwise you get a warning):
> > >
> > >
> > > mUIThreadDone = false;
> > > cordova.getActivity().runOnUiThread(new Runnable() {
> > > public void run() {
> > > CordovaWebView uiThreadView = webView;
> > > picture = uiThreadView.capturePicture();
> > > mUIThreadDone = true;
> > > }
> > > });
> > > //rest of the work is done on core thread
> > > while(!mUIThreadDone) {}
> > > //write the picture to a file
> > > ...
> > >
> > > I could do the file io and compares on a different thread than the
> > > core one, but for the major use case, the core thread needs to wait no
> > > matter what to ensure that the file is written and the comparison is
> > > complete, so I wouldn't expect much gain there.
> > >
> > >
> > > However, I've been experimenting with a more threaded version that
> > > does the file io and comparison on a background thread as you
> > > suggested. If you have time please check it out. While the code is
> > > not as clean, I think it addresses the issues you raised Andrew. For
> > > the major use case, performance is still the same as expected, but if
> > > we want pure throughput on captures there's a big gain from about 3
> > > captures per second (cps) to 10 cps on a Nexus 10.
> > >
> > >
> > > Here's the new plugin, with moar threading!
> > >
> > >
> > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
> > >
> > >
> > > Once again, thanks for all your feedback on this, I haven't had much
> > > experience with threading in java before now.
> > >
> > >
> > > -Aaron
> > >
> > >
> > >
> > >
> > >
> > > On Thu, Mar 14, 2013 at 8:10 AM, Andrew Grieve <ag...@chromium.org>
> > > wrote:
> > >
> > > > My suggestion was just to prevent the UI thread from locking up so
> that
> > > you
> > > > app doesn't appear to be frozen when the plugin is doing its thing.
> > > >
> > > >
> > > > On Wed, Mar 13, 2013 at 5:03 PM, Aaron Charbonneau <
> amcharbo@gmail.com
> > > > >wrote:
> > > >
> > > > > Yah that's how I have it working now :) No problem with that
> approach
> > > for
> > > > > working with Jasmine.
> > > > >
> > > > > I made reference to running them asynchronously to respond to
> Andrew
> > > > about
> > > > > returning immediately after the bits are captured, which in the
> case
> > of
> > > > > using that with Jasmine would only see a benefit IF we could run
> > > multiple
> > > > > 'it' blocks at the same time.
> > > > >
> > > > > I think however Andrew was thinking of a different use case where
> we
> > > > aren't
> > > > > limited by a framework that requires tests to run serially, and
> > results
> > > > can
> > > > > just come back as they are ready, and be evaluated all together at
> > the
> > > > end,
> > > > > or not at all if you just want the captures and no evaluation.
> > > > >
> > > > > -Aaron
> > > > >
> > > > >
> > > > > On Wed, Mar 13, 2013 at 1:53 PM, Braden Shepherdson <
> > > braden@chromium.org
> > > > > >wrote:
> > > > >
> > > > > > As far as I know, you're correct: there's no way to move on the
> > next
> > > > it()
> > > > > > block before the first one is done.
> > > > > >
> > > > > > What's the problem with just making the tests wait? Why does the
> > next
> > > > > it()
> > > > > > block need to start early? You can make your call to Java, use
> > > waitFor
> > > > to
> > > > > > wait until the callback with the data is called, run your
> > > expectations,
> > > > > and
> > > > > > then move on to the next it() block. If the next it() block is
> > using
> > > > the
> > > > > > result of this one, then they should be one it() block.
> > > > > >
> > > > > > Braden
> > > > > >
> > > > > >
> > > > > > On Wed, Mar 13, 2013 at 4:46 PM, Aaron Charbonneau <
> > > amcharbo@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > Hi Braden,
> > > > > > > Sorry maybe I am overlooking something. The way I understand
> it
> > is
> > > > > that
> > > > > > a
> > > > > > > previous 'it' block needs to complete before the next one can
> > run.
> > > > So
> > > > > I
> > > > > > > have various tests setup like this:
> > > > > > >
> > > > > > > it("renders something", function() {
> > > > > > > //draw something
> > > > > > > ....
> > > > > > > captureAndCompare(); //call the native function
> > > > > > > waitsFor(function() {
> > > > > > > return captureComplete; //set by the native function callback
> > > > > > > }, "capture never completed", 10000);
> > > > > > > runs(function() {
> > > > > > > expect(captureResult).toBe(0); //expects difference to be 0
> > > > > > > });
> > > > > > > });
> > > > > > > //next 'it' test
> > > > > > > ...
> > > > > > >
> > > > > > > The expect block needs to wait until there is a result returned
> > > from
> > > > > the
> > > > > > > native captureAndCompare call. Therefore we use a
> waitsFor/runs
> > > > block
> > > > > > > there.
> > > > > > >
> > > > > > > But say we wanted to allow Javascript to run immediately after
> > > > calling
> > > > > > that
> > > > > > > native function. The only thing you need to wait for is that
> > Java
> > > > > > grabbed
> > > > > > > the bits, and the rest of the processing will be done 'at some
> > > point
> > > > in
> > > > > > the
> > > > > > > future.' So as Andrew suggested you would return as soon as
> the
> > > bits
> > > > > are
> > > > > > > captured, and allow JS to run again. The problem is that our
> > > expect
> > > > > > block
> > > > > > > needs the results, so that has to wait, the only other thing to
> > do
> > > > > would
> > > > > > be
> > > > > > > to run the next 'it' block.
> > > > > > >
> > > > > > > This is the part I didn't think was possible, can the next 'it'
> > > block
> > > > > be
> > > > > > > run while the previous 'it' is still waiting to execute the
> > > 'expect'
> > > > > > block?
> > > > > > > Or is there another approach all together that would allow
> this?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Aaron
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <
> > > > > braden@chromium.org
> > > > > > > >wrote:
> > > > > > >
> > > > > > > > Jasmine does support asynchronous tests, and waiting an
> > arbitrary
> > > > > > period
> > > > > > > or
> > > > > > > > until some condition is true. Why does that not work for
> these
> > > > tests?
> > > > > > > >
> > > > > > > > Braden
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <
> > > > > amcharbo@gmail.com
> > > > > > > > >wrote:
> > > > > > > >
> > > > > > > > > Quick follow up question for the return values stuff:
> > > > > > > > > Does it make sense to have my functions called from execute
> > > > > > (capture()
> > > > > > > > and
> > > > > > > > > captureAndCompare()) return an error string if one is
> > > encountered
> > > > > and
> > > > > > > > pass
> > > > > > > > > that back in callbackContext.error, then have execute
> return
> > > > true?
> > > > > > > > >
> > > > > > > > > Or is there a preferred way to get errors from sub
> functions
> > > back
> > > > > > into
> > > > > > > > > Javascript?
> > > > > > > > >
> > > > > > > > > Thanks!
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <
> > > > > > > amcharbo@gmail.com
> > > > > > > > > >wrote:
> > > > > > > > >
> > > > > > > > > > Thanks Andrew these are great suggestions!
> > > > > > > > > >
> > > > > > > > > > About not needing the busy-wait in getScreenBits:
> > > > > > > > > > I'm not a fan of busy-waiting either.
> > > > > > > > > > Perhaps it's a side effect of Jasmine requiring tests to
> be
> > > run
> > > > > and
> > > > > > > > > > evaluated synchronously that I decided to use a busy-wait
> > in
> > > > > > > > > getScreenBits.
> > > > > > > > > > If the Runnable calls CallbackContext.success/error once
> > the
> > > > > bits
> > > > > > > are
> > > > > > > > > > captured (but not written to file) on the UIThread, the
> > > > > Javascript
> > > > > > > side
> > > > > > > > > > will proceed to run before it has an image url/compare
> > > result,
> > > > > most
> > > > > > > > > likely
> > > > > > > > > > causing that test to fail. As far as I know Jasmine
> > doesn't
> > > > > allow
> > > > > > > for
> > > > > > > > > > kicking off a bunch of tests and then gather all the
> > results
> > > at
> > > > > the
> > > > > > > end
> > > > > > > > > and
> > > > > > > > > > output pass/fails (each 'expect' function needs to be
> > within
> > > an
> > > > > > 'it'
> > > > > > > > > block,
> > > > > > > > > > essentially that means one test must be evaluated before
> > the
> > > > next
> > > > > > can
> > > > > > > > > > begin).
> > > > > > > > > >
> > > > > > > > > > I can see the usefulness of allowing the capture, file
> io,
> > > and
> > > > > > > > comparison
> > > > > > > > > > to be able to run asynchronously from the Javascript
> > though.
> > > In
> > > > > the
> > > > > > > > case
> > > > > > > > > > where you have your own test framework that CAN kick off
> a
> > > > bunch
> > > > > of
> > > > > > > > tests
> > > > > > > > > > and just gather the results as they come in and do a
> batch
> > > > > > evaluation
> > > > > > > > at
> > > > > > > > > > the end, or in the case where you don't want to do any
> > > > evaluation
> > > > > > in
> > > > > > > > your
> > > > > > > > > > app and just want to get captures as fast as you can,
> > > > > asynchronous
> > > > > > > > could
> > > > > > > > > > increase performance there.
> > > > > > > > > >
> > > > > > > > > > I could try to implement it this way too, if you think
> > > that's a
> > > > > > worth
> > > > > > > > > > while use case, maybe just provide an 'asynchronous' flag
> > in
> > > > > > > > > CaptureOptions?
> > > > > > > > > >
> > > > > > > > > > For doing the fileIO/compare using cordova.getThreadPool,
> > is
> > > > > there
> > > > > > a
> > > > > > > > > > benefit to doing that if I still plan having the
> operation
> > > > block
> > > > > > the
> > > > > > > > > > Javascript? For asynchronous mode I see why that makes
> > > sense.
> > > > > > > > > >
> > > > > > > > > > For the return values, I'll make those changes right
> away!
> > > > > Thanks
> > > > > > > > again
> > > > > > > > > > for providing your insight, hopefully I understood it
> > > > correctly.
> > > > > > > > > >
> > > > > > > > > > -Aaron
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <
> > > > > > agrieve@chromium.org
> > > > > > > > > >wrote:
> > > > > > > > > >
> > > > > > > > > >> Had a glance at your code. Things you may want to
> address:
> > > > > > > > > >>
> > > > > > > > > >> while(!mUIThreadDone) {}
> > > > > > > > > >>
> > > > > > > > > >> You shouldn't need to busy-wait. Pass the
> CallbackContext
> > to
> > > > the
> > > > > > > > > Runnable,
> > > > > > > > > >> and call success/error on it whenever you're done. No
> need
> > > to
> > > > > have
> > > > > > > the
> > > > > > > > > web
> > > > > > > > > >> thread wait for it. Even better would be to use a second
> > > > > Runnable
> > > > > > to
> > > > > > > > do
> > > > > > > > > >> the
> > > > > > > > > >> image compare / file IO on a background thread via
> > > > > > > > > cordova.getThreadPool()
> > > > > > > > > >>
> > > > > > > > > >> The return value to execute() should only be false if an
> > > > invalid
> > > > > > > > command
> > > > > > > > > >> was passed. It's not meant to indicate the success of
> the
> > > > > > operation.
> > > > > > > > You
> > > > > > > > > >> should be returning true for it.
> > > > > > > > > >>
> > > > > > > > > >> Instead of return "success"/"", it might simplify
> things a
> > > bit
> > > > > to
> > > > > > > > > return a
> > > > > > > > > >> boolean of true/false
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <
> > > > > > > lorin.beer.dev@gmail.com
> > > > > > > > > >> >wrote:
> > > > > > > > > >>
> > > > > > > > > >> > all the same, very impressive and faster then I
> > expected.
> > > > > > > > > >> >
> > > > > > > > > >> > - Lorin
> > > > > > > > > >> >
> > > > > > > > > >> >
> > > > > > > > > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <
> > > > > > > > > amcharbo@gmail.com
> > > > > > > > > >> > >wrote:
> > > > > > > > > >> >
> > > > > > > > > >> > > Thanks Lorin.
> > > > > > > > > >> > >
> > > > > > > > > >> > > I did a quick test for throughput on an animating
> > > canvas.
> > > > I
> > > > > > was
> > > > > > > > > able
> > > > > > > > > >> to
> > > > > > > > > >> > > grab 30 frames within a 5 seconds which equates to
> > > around
> > > > > > 166ms
> > > > > > > > per
> > > > > > > > > >> > capture
> > > > > > > > > >> > > on a Galaxy Nexus. Unfortunately not fast enough to
> > > > > automate
> > > > > > > > > >> something
> > > > > > > > > >> > > like video. As with any kind of capturing system,
> > often
> > > > the
> > > > > > > > > overhead
> > > > > > > > > >> of
> > > > > > > > > >> > > copying bits around is enough to slow down the
> > operation
> > > > of
> > > > > > the
> > > > > > > > app.
> > > > > > > > > >> > >
> > > > > > > > > >> > > -Aaron
> > > > > > > > > >> > >
> > > > > > > > > >> > >
> > > > > > > > > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
> > > > > > > > > >> lorin.beer.dev@gmail.com
> > > > > > > > > >> > > >wrote:
> > > > > > > > > >> > >
> > > > > > > > > >> > > > Hey Aaron,
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > very cool stuff, looking forward to checking it
> out!
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > Question: any performance hit on the app by using
> > > > Capture?
> > > > > > > > What's
> > > > > > > > > >> the
> > > > > > > > > >> > > > expected throughput on images taken in this way
> > > (given a
> > > > > > > > > particular
> > > > > > > > > >> > > device,
> > > > > > > > > >> > > > say Galaxy Nexus)?
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > - Lorin
> > > > > > > > > >> > > >
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron
> Charbonneau
> > <
> > > > > > > > > >> > amcharbo@gmail.com
> > > > > > > > > >> > > > >wrote:
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > > Glad you like it :)
> > > > > > > > > >> > > > > Yes in fact the actual capture makes use of
> > > > > > > > > View.capturePicture()
> > > > > > > > > >> > which
> > > > > > > > > >> > > > > actually grabs the entire document, then that
> can
> > be
> > > > > > clipped
> > > > > > > > > down
> > > > > > > > > >> to
> > > > > > > > > >> > > the
> > > > > > > > > >> > > > > size/location of a specific element.
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > -Aaron
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <
> > > > > > > > > >> mmocny@chromium.org>
> > > > > > > > > >> > > > > wrote:
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > > Aaron,
> > > > > > > > > >> > > > > >
> > > > > > > > > >> > > > > > I haven't even begun looking at your
> > > implementation,
> > > > > but
> > > > > > > I'm
> > > > > > > > > >> just
> > > > > > > > > >> > > going
> > > > > > > > > >> > > > > to
> > > > > > > > > >> > > > > > say it: this is awesome!
> > > > > > > > > >> > > > > >
> > > > > > > > > >> > > > > > First question: When Capturing a DOM element,
> > can
> > > > you
> > > > > > > > capture
> > > > > > > > > >> > 'body'
> > > > > > > > > >> > > to
> > > > > > > > > >> > > > > > grab it&children for a "full content
> > screenshot",
> > > or
> > > > > > does
> > > > > > > it
> > > > > > > > > >> have
> > > > > > > > > >> > to
> > > > > > > > > >> > > > be a
> > > > > > > > > >> > > > > > specific single element?
> > > > > > > > > >> > > > > >
> > > > > > > > > >> > > > > > -Michal
> > > > > > > > > >> > > > > >
> > > > > > > > > >> > > > > >
> > > > > > > > > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron
> > > Charbonneau <
> > > > > > > > > >> > > amcharbo@gmail.com
> > > > > > > > > >> > > > > > >wrote:
> > > > > > > > > >> > > > > >
> > > > > > > > > >> > > > > > > Greetings! My name is Aaron Charbonneau,
> > happy
> > > to
> > > > > be
> > > > > > a
> > > > > > > > new
> > > > > > > > > >> > member
> > > > > > > > > >> > > of
> > > > > > > > > >> > > > > the
> > > > > > > > > >> > > > > > > mailing list!
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > I have been developing a sceenshot plugin
> for
> > > > > Cordova
> > > > > > to
> > > > > > > > > help
> > > > > > > > > >> > > > > facilitate
> > > > > > > > > >> > > > > > > automation testing and debugging of Cordova
> > > apps,
> > > > > and
> > > > > > I
> > > > > > > > > would
> > > > > > > > > >> > love
> > > > > > > > > >> > > > some
> > > > > > > > > >> > > > > > > feedback on it. Currently Cordova provides
> a
> > > > bunch
> > > > > of
> > > > > > > > > native
> > > > > > > > > >> > > > functions
> > > > > > > > > >> > > > > > > that allow you to do some cool stuff, but
> not
> > > much
> > > > > > > > > >> functionality
> > > > > > > > > >> > to
> > > > > > > > > >> > > > > test
> > > > > > > > > >> > > > > > > the apps that make use of them. Being able
> to
> > > > take
> > > > > a
> > > > > > > > > capture
> > > > > > > > > >> of
> > > > > > > > > >> > > the
> > > > > > > > > >> > > > > > screen
> > > > > > > > > >> > > > > > > from within you app is a great way to
> automate
> > > > > testing
> > > > > > > or
> > > > > > > > > get
> > > > > > > > > >> > > > > additional
> > > > > > > > > >> > > > > > > information for debugging an issue. Since
> > there
> > > > is
> > > > > no
> > > > > > > > > >> > > > > > > Javascript mechanism for taking screen
> > captures
> > > > the
> > > > > > > > solution
> > > > > > > > > >> > would
> > > > > > > > > >> > > > have
> > > > > > > > > >> > > > > > to
> > > > > > > > > >> > > > > > > be native, which fits nicely into the "gap"
> > that
> > > > > > > > > >> Cordova/Phonegap
> > > > > > > > > >> > > > > > bridges.
> > > > > > > > > >> > > > > > > Any medium to large scale app can benefit
> > > greatly
> > > > > > from
> > > > > > > > > >> > automation
> > > > > > > > > >> > > > > > testing
> > > > > > > > > >> > > > > > > and any app can benefit from an extra
> > debugging
> > > > > tool,
> > > > > > > and
> > > > > > > > > >> that is
> > > > > > > > > >> > > > what
> > > > > > > > > >> > > > > I
> > > > > > > > > >> > > > > > > hope this screenshot plugin can help
> achieve.
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > Currently the plugin offers 2 functions:
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > Capture():
> > > > > > > > > >> > > > > > > * Take a capture of the current view, write
> > that
> > > > > > capture
> > > > > > > > to
> > > > > > > > > a
> > > > > > > > > >> > .png
> > > > > > > > > >> > > > file
> > > > > > > > > >> > > > > > > with the specified file name and sub
> directory
> > > of
> > > > > the
> > > > > > > > sdcard
> > > > > > > > > >> > > > (fallback
> > > > > > > > > >> > > > > to
> > > > > > > > > >> > > > > > > emulated sdcard in the case there isn't an
> > > sdcard
> > > > > > > mounted)
> > > > > > > > > >> > > > > > > * Able to create a sub-screenshot with a
> > > specified
> > > > > > > > rectangle
> > > > > > > > > >> in
> > > > > > > > > >> > > order
> > > > > > > > > >> > > > > to
> > > > > > > > > >> > > > > > > block out ui elements that may be variable,
> > and
> > > > also
> > > > > > > save
> > > > > > > > > >> space.
> > > > > > > > > >> > > > > > > * Can take captures of images/dom elements
> > > > > (including
> > > > > > > > > canvas)
> > > > > > > > > >> > that
> > > > > > > > > >> > > > are
> > > > > > > > > >> > > > > > > lager than the actual screen size
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > CaptureAndCompare():
> > > > > > > > > >> > > > > > > * All the functionality of Capture()
> > > > > > > > > >> > > > > > > * Perform a comparison between the captured
> > > image
> > > > > and
> > > > > > a
> > > > > > > > > >> baseline
> > > > > > > > > >> > > > image
> > > > > > > > > >> > > > > > > located at the specified location in either
> > the
> > > > > assets
> > > > > > > > > folder
> > > > > > > > > >> or
> > > > > > > > > >> > > the
> > > > > > > > > >> > > > > > > sdcard.
> > > > > > > > > >> > > > > > > * User can specify per color channel
> > tolerances
> > > as
> > > > > > well
> > > > > > > as
> > > > > > > > > >> total
> > > > > > > > > >> > > > pixel
> > > > > > > > > >> > > > > > > tolerances to avoid false positives for the
> > > > > inevitable
> > > > > > > > > >> rendering
> > > > > > > > > >> > > > > > > differences across devices.
> > > > > > > > > >> > > > > > > * Optionally output a png file that contains
> > the
> > > > > > > > differences
> > > > > > > > > >> > > between
> > > > > > > > > >> > > > > the
> > > > > > > > > >> > > > > > > actual and the baseline for debugging/triage
> > > > > purposes,
> > > > > > > two
> > > > > > > > > >> modes:
> > > > > > > > > >> > > > > binary
> > > > > > > > > >> > > > > > > diff (all failing pixels appear as solid
> > white)
> > > or
> > > > > the
> > > > > > > > true
> > > > > > > > > >> > > > differences
> > > > > > > > > >> > > > > > > between pixels.
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > If you can spare some time, I would love it
> if
> > > you
> > > > > > could
> > > > > > > > > take
> > > > > > > > > >> a
> > > > > > > > > >> > > look
> > > > > > > > > >> > > > at
> > > > > > > > > >> > > > > > the
> > > > > > > > > >> > > > > > > api and parameters I have defined to make
> sure
> > > > they
> > > > > > > adhere
> > > > > > > > > to
> > > > > > > > > >> > > Cordova
> > > > > > > > > >> > > > > > > plugin best practices. The most crucial
> part
> > > > would
> > > > > be
> > > > > > > in
> > > > > > > > > the
> > > > > > > > > >> > > plugin
> > > > > > > > > >> > > > > > itself
> > > > > > > > > >> > > > > > > at ScreenCapture.java:
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > >
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > >
> > > > > > > > > >> > >
> > > > > > > > > >> >
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > > > > > > > >> > > > > > > and captureAndCompare() are the two exposed
> > > > > functions)
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > I'm also interested to know you thoughts on
> > it's
> > > > > > usage.
> > > > > > > I
> > > > > > > > > >> > started
> > > > > > > > > >> > > > with
> > > > > > > > > >> > > > > > the
> > > > > > > > > >> > > > > > > mobile-spec testing framework and put some
> > quick
> > > > > > > rendering
> > > > > > > > > >> tests
> > > > > > > > > >> > > into
> > > > > > > > > >> > > > > the
> > > > > > > > > >> > > > > > > Autotest section as a viable use case. In
> > order
> > > > to
> > > > > > get
> > > > > > > > the
> > > > > > > > > >> > WebView
> > > > > > > > > >> > > > > into
> > > > > > > > > >> > > > > > > the state I wanted to capture, I had to
> > > implement
> > > > a
> > > > > > wait
> > > > > > > > > >> timer to
> > > > > > > > > >> > > > allow
> > > > > > > > > >> > > > > > the
> > > > > > > > > >> > > > > > > view to update before taking the capture.
> > Once
> > > > the
> > > > > > wait
> > > > > > > > > >> function
> > > > > > > > > >> > > > > > > completes, the capture can be taken and
> > > everything
> > > > > > from
> > > > > > > > > there
> > > > > > > > > >> is
> > > > > > > > > >> > > > > callback
> > > > > > > > > >> > > > > > > based. I use the Jasmine waitsFor()/runs()
> > > blocks
> > > > > to
> > > > > > > make
> > > > > > > > > the
> > > > > > > > > >> > > tests
> > > > > > > > > >> > > > > run
> > > > > > > > > >> > > > > > > synchronously. For some validation testing
> I
> > > > made a
> > > > > > > > compare
> > > > > > > > > >> > > function
> > > > > > > > > >> > > > > in
> > > > > > > > > >> > > > > > > Javascript to run against the native
> compare.
> > > > Turns
> > > > > > out
> > > > > > > > > Java
> > > > > > > > > >> > runs
> > > > > > > > > >> > > > alot
> > > > > > > > > >> > > > > > > faster than Javascript (surprise!) thus the
> > > native
> > > > > > > compare
> > > > > > > > > is
> > > > > > > > > >> > about
> > > > > > > > > >> > > > > 2-5x
> > > > > > > > > >> > > > > > > faster. All in all the process is fairly
> > quick,
> > > > the
> > > > > > > full
> > > > > > > > > >> > > > > > captureAndCompare
> > > > > > > > > >> > > > > > > with 3 file io's takes about 233ms on a
> Nexus
> > > One,
> > > > > and
> > > > > > > > > ~100ms
> > > > > > > > > >> on
> > > > > > > > > >> > a
> > > > > > > > > >> > > > > Nexus
> > > > > > > > > >> > > > > > 10
> > > > > > > > > >> > > > > > > (256x256 image size).
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > Anyways here is the usage:
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > >
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > >
> > > > > > > > > >> > >
> > > > > > > > > >> >
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > And here's the JS wrapper for the plugin
> > calls:
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > >
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > >
> > > > > > > > > >> > >
> > > > > > > > > >> >
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > Thanks for your time, and I look forward to
> > ANY
> > > > > > feedback
> > > > > > > > > >> positive
> > > > > > > > > >> > > or
> > > > > > > > > >> > > > > > > negative.
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > Thanks,
> > > > > > > > > >> > > > > > > Aaron
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > >
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > >
> > > > > > > > > >> > >
> > > > > > > > > >> >
> > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Aaron Charbonneau <am...@gmail.com>.
"You could avoid the blocking queue if you dispatch the background thread's
runnable at the end of the UI thread's block."
-Great idea, done.
I traced through to see what sending a NO_RESULT PluginResult with
keepCallback==true does, and it looks like a no-op
(NativeToJsMessageQueue.java, addPluginResult()):
boolean noResult = result.getStatus() ==
PluginResult.Status.NO_RESULT.ordinal();
boolean keepCallback = result.getKeepCallback();
if (noResult && keepCallback) {
return;
}
I'm sure I'm missing something here, is that really a no-op? Did you
mean for me to send a PluginResult.Status.OK with keepCallBack=true?
Thanks,
Aaron
On Sat, Mar 16, 2013 at 6:15 AM, Andrew Grieve <ag...@chromium.org> wrote:
> You could avoid the blocking queue if you dispatch the background thread's
> runnable at the end of the UI thread's block.
>
> I see you send a success callback as soon as the image is captured, but
> before it is saved. This would improve throughput, but it might be even
> more useful if you set the keepCallback flag on it and then send another
> success response once you have the filename / comparison results. It's the
> filename that the app is probably interested in.
>
>
> On Fri, Mar 15, 2013 at 9:41 PM, Aaron Charbonneau <amcharbo@gmail.com
> >wrote:
>
> > Ah thanks for clarifying that.
> >
> > I'm not sure how I can alleviate the ui thread any more than I am
> > currently. It is already doing the bare minimum amount of work on the
> > there. Right now the only work done on the ui thread is
> > view.capturePicture() which is the recommended thread for doing that work
> > (otherwise you get a warning):
> >
> >
> > mUIThreadDone = false;
> > cordova.getActivity().runOnUiThread(new Runnable() {
> > public void run() {
> > CordovaWebView uiThreadView = webView;
> > picture = uiThreadView.capturePicture();
> > mUIThreadDone = true;
> > }
> > });
> > //rest of the work is done on core thread
> > while(!mUIThreadDone) {}
> > //write the picture to a file
> > ...
> >
> > I could do the file io and compares on a different thread than the
> > core one, but for the major use case, the core thread needs to wait no
> > matter what to ensure that the file is written and the comparison is
> > complete, so I wouldn't expect much gain there.
> >
> >
> > However, I've been experimenting with a more threaded version that
> > does the file io and comparison on a background thread as you
> > suggested. If you have time please check it out. While the code is
> > not as clean, I think it addresses the issues you raised Andrew. For
> > the major use case, performance is still the same as expected, but if
> > we want pure throughput on captures there's a big gain from about 3
> > captures per second (cps) to 10 cps on a Nexus 10.
> >
> >
> > Here's the new plugin, with moar threading!
> >
> >
> >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
> >
> >
> > Once again, thanks for all your feedback on this, I haven't had much
> > experience with threading in java before now.
> >
> >
> > -Aaron
> >
> >
> >
> >
> >
> > On Thu, Mar 14, 2013 at 8:10 AM, Andrew Grieve <ag...@chromium.org>
> > wrote:
> >
> > > My suggestion was just to prevent the UI thread from locking up so that
> > you
> > > app doesn't appear to be frozen when the plugin is doing its thing.
> > >
> > >
> > > On Wed, Mar 13, 2013 at 5:03 PM, Aaron Charbonneau <amcharbo@gmail.com
> > > >wrote:
> > >
> > > > Yah that's how I have it working now :) No problem with that approach
> > for
> > > > working with Jasmine.
> > > >
> > > > I made reference to running them asynchronously to respond to Andrew
> > > about
> > > > returning immediately after the bits are captured, which in the case
> of
> > > > using that with Jasmine would only see a benefit IF we could run
> > multiple
> > > > 'it' blocks at the same time.
> > > >
> > > > I think however Andrew was thinking of a different use case where we
> > > aren't
> > > > limited by a framework that requires tests to run serially, and
> results
> > > can
> > > > just come back as they are ready, and be evaluated all together at
> the
> > > end,
> > > > or not at all if you just want the captures and no evaluation.
> > > >
> > > > -Aaron
> > > >
> > > >
> > > > On Wed, Mar 13, 2013 at 1:53 PM, Braden Shepherdson <
> > braden@chromium.org
> > > > >wrote:
> > > >
> > > > > As far as I know, you're correct: there's no way to move on the
> next
> > > it()
> > > > > block before the first one is done.
> > > > >
> > > > > What's the problem with just making the tests wait? Why does the
> next
> > > > it()
> > > > > block need to start early? You can make your call to Java, use
> > waitFor
> > > to
> > > > > wait until the callback with the data is called, run your
> > expectations,
> > > > and
> > > > > then move on to the next it() block. If the next it() block is
> using
> > > the
> > > > > result of this one, then they should be one it() block.
> > > > >
> > > > > Braden
> > > > >
> > > > >
> > > > > On Wed, Mar 13, 2013 at 4:46 PM, Aaron Charbonneau <
> > amcharbo@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Hi Braden,
> > > > > > Sorry maybe I am overlooking something. The way I understand it
> is
> > > > that
> > > > > a
> > > > > > previous 'it' block needs to complete before the next one can
> run.
> > > So
> > > > I
> > > > > > have various tests setup like this:
> > > > > >
> > > > > > it("renders something", function() {
> > > > > > //draw something
> > > > > > ....
> > > > > > captureAndCompare(); //call the native function
> > > > > > waitsFor(function() {
> > > > > > return captureComplete; //set by the native function callback
> > > > > > }, "capture never completed", 10000);
> > > > > > runs(function() {
> > > > > > expect(captureResult).toBe(0); //expects difference to be 0
> > > > > > });
> > > > > > });
> > > > > > //next 'it' test
> > > > > > ...
> > > > > >
> > > > > > The expect block needs to wait until there is a result returned
> > from
> > > > the
> > > > > > native captureAndCompare call. Therefore we use a waitsFor/runs
> > > block
> > > > > > there.
> > > > > >
> > > > > > But say we wanted to allow Javascript to run immediately after
> > > calling
> > > > > that
> > > > > > native function. The only thing you need to wait for is that
> Java
> > > > > grabbed
> > > > > > the bits, and the rest of the processing will be done 'at some
> > point
> > > in
> > > > > the
> > > > > > future.' So as Andrew suggested you would return as soon as the
> > bits
> > > > are
> > > > > > captured, and allow JS to run again. The problem is that our
> > expect
> > > > > block
> > > > > > needs the results, so that has to wait, the only other thing to
> do
> > > > would
> > > > > be
> > > > > > to run the next 'it' block.
> > > > > >
> > > > > > This is the part I didn't think was possible, can the next 'it'
> > block
> > > > be
> > > > > > run while the previous 'it' is still waiting to execute the
> > 'expect'
> > > > > block?
> > > > > > Or is there another approach all together that would allow this?
> > > > > >
> > > > > > Thanks,
> > > > > > Aaron
> > > > > >
> > > > > >
> > > > > > On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <
> > > > braden@chromium.org
> > > > > > >wrote:
> > > > > >
> > > > > > > Jasmine does support asynchronous tests, and waiting an
> arbitrary
> > > > > period
> > > > > > or
> > > > > > > until some condition is true. Why does that not work for these
> > > tests?
> > > > > > >
> > > > > > > Braden
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <
> > > > amcharbo@gmail.com
> > > > > > > >wrote:
> > > > > > >
> > > > > > > > Quick follow up question for the return values stuff:
> > > > > > > > Does it make sense to have my functions called from execute
> > > > > (capture()
> > > > > > > and
> > > > > > > > captureAndCompare()) return an error string if one is
> > encountered
> > > > and
> > > > > > > pass
> > > > > > > > that back in callbackContext.error, then have execute return
> > > true?
> > > > > > > >
> > > > > > > > Or is there a preferred way to get errors from sub functions
> > back
> > > > > into
> > > > > > > > Javascript?
> > > > > > > >
> > > > > > > > Thanks!
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <
> > > > > > amcharbo@gmail.com
> > > > > > > > >wrote:
> > > > > > > >
> > > > > > > > > Thanks Andrew these are great suggestions!
> > > > > > > > >
> > > > > > > > > About not needing the busy-wait in getScreenBits:
> > > > > > > > > I'm not a fan of busy-waiting either.
> > > > > > > > > Perhaps it's a side effect of Jasmine requiring tests to be
> > run
> > > > and
> > > > > > > > > evaluated synchronously that I decided to use a busy-wait
> in
> > > > > > > > getScreenBits.
> > > > > > > > > If the Runnable calls CallbackContext.success/error once
> the
> > > > bits
> > > > > > are
> > > > > > > > > captured (but not written to file) on the UIThread, the
> > > > Javascript
> > > > > > side
> > > > > > > > > will proceed to run before it has an image url/compare
> > result,
> > > > most
> > > > > > > > likely
> > > > > > > > > causing that test to fail. As far as I know Jasmine
> doesn't
> > > > allow
> > > > > > for
> > > > > > > > > kicking off a bunch of tests and then gather all the
> results
> > at
> > > > the
> > > > > > end
> > > > > > > > and
> > > > > > > > > output pass/fails (each 'expect' function needs to be
> within
> > an
> > > > > 'it'
> > > > > > > > block,
> > > > > > > > > essentially that means one test must be evaluated before
> the
> > > next
> > > > > can
> > > > > > > > > begin).
> > > > > > > > >
> > > > > > > > > I can see the usefulness of allowing the capture, file io,
> > and
> > > > > > > comparison
> > > > > > > > > to be able to run asynchronously from the Javascript
> though.
> > In
> > > > the
> > > > > > > case
> > > > > > > > > where you have your own test framework that CAN kick off a
> > > bunch
> > > > of
> > > > > > > tests
> > > > > > > > > and just gather the results as they come in and do a batch
> > > > > evaluation
> > > > > > > at
> > > > > > > > > the end, or in the case where you don't want to do any
> > > evaluation
> > > > > in
> > > > > > > your
> > > > > > > > > app and just want to get captures as fast as you can,
> > > > asynchronous
> > > > > > > could
> > > > > > > > > increase performance there.
> > > > > > > > >
> > > > > > > > > I could try to implement it this way too, if you think
> > that's a
> > > > > worth
> > > > > > > > > while use case, maybe just provide an 'asynchronous' flag
> in
> > > > > > > > CaptureOptions?
> > > > > > > > >
> > > > > > > > > For doing the fileIO/compare using cordova.getThreadPool,
> is
> > > > there
> > > > > a
> > > > > > > > > benefit to doing that if I still plan having the operation
> > > block
> > > > > the
> > > > > > > > > Javascript? For asynchronous mode I see why that makes
> > sense.
> > > > > > > > >
> > > > > > > > > For the return values, I'll make those changes right away!
> > > > Thanks
> > > > > > > again
> > > > > > > > > for providing your insight, hopefully I understood it
> > > correctly.
> > > > > > > > >
> > > > > > > > > -Aaron
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <
> > > > > agrieve@chromium.org
> > > > > > > > >wrote:
> > > > > > > > >
> > > > > > > > >> Had a glance at your code. Things you may want to address:
> > > > > > > > >>
> > > > > > > > >> while(!mUIThreadDone) {}
> > > > > > > > >>
> > > > > > > > >> You shouldn't need to busy-wait. Pass the CallbackContext
> to
> > > the
> > > > > > > > Runnable,
> > > > > > > > >> and call success/error on it whenever you're done. No need
> > to
> > > > have
> > > > > > the
> > > > > > > > web
> > > > > > > > >> thread wait for it. Even better would be to use a second
> > > > Runnable
> > > > > to
> > > > > > > do
> > > > > > > > >> the
> > > > > > > > >> image compare / file IO on a background thread via
> > > > > > > > cordova.getThreadPool()
> > > > > > > > >>
> > > > > > > > >> The return value to execute() should only be false if an
> > > invalid
> > > > > > > command
> > > > > > > > >> was passed. It's not meant to indicate the success of the
> > > > > operation.
> > > > > > > You
> > > > > > > > >> should be returning true for it.
> > > > > > > > >>
> > > > > > > > >> Instead of return "success"/"", it might simplify things a
> > bit
> > > > to
> > > > > > > > return a
> > > > > > > > >> boolean of true/false
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <
> > > > > > lorin.beer.dev@gmail.com
> > > > > > > > >> >wrote:
> > > > > > > > >>
> > > > > > > > >> > all the same, very impressive and faster then I
> expected.
> > > > > > > > >> >
> > > > > > > > >> > - Lorin
> > > > > > > > >> >
> > > > > > > > >> >
> > > > > > > > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <
> > > > > > > > amcharbo@gmail.com
> > > > > > > > >> > >wrote:
> > > > > > > > >> >
> > > > > > > > >> > > Thanks Lorin.
> > > > > > > > >> > >
> > > > > > > > >> > > I did a quick test for throughput on an animating
> > canvas.
> > > I
> > > > > was
> > > > > > > > able
> > > > > > > > >> to
> > > > > > > > >> > > grab 30 frames within a 5 seconds which equates to
> > around
> > > > > 166ms
> > > > > > > per
> > > > > > > > >> > capture
> > > > > > > > >> > > on a Galaxy Nexus. Unfortunately not fast enough to
> > > > automate
> > > > > > > > >> something
> > > > > > > > >> > > like video. As with any kind of capturing system,
> often
> > > the
> > > > > > > > overhead
> > > > > > > > >> of
> > > > > > > > >> > > copying bits around is enough to slow down the
> operation
> > > of
> > > > > the
> > > > > > > app.
> > > > > > > > >> > >
> > > > > > > > >> > > -Aaron
> > > > > > > > >> > >
> > > > > > > > >> > >
> > > > > > > > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
> > > > > > > > >> lorin.beer.dev@gmail.com
> > > > > > > > >> > > >wrote:
> > > > > > > > >> > >
> > > > > > > > >> > > > Hey Aaron,
> > > > > > > > >> > > >
> > > > > > > > >> > > > very cool stuff, looking forward to checking it out!
> > > > > > > > >> > > >
> > > > > > > > >> > > > Question: any performance hit on the app by using
> > > Capture?
> > > > > > > What's
> > > > > > > > >> the
> > > > > > > > >> > > > expected throughput on images taken in this way
> > (given a
> > > > > > > > particular
> > > > > > > > >> > > device,
> > > > > > > > >> > > > say Galaxy Nexus)?
> > > > > > > > >> > > >
> > > > > > > > >> > > > - Lorin
> > > > > > > > >> > > >
> > > > > > > > >> > > >
> > > > > > > > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau
> <
> > > > > > > > >> > amcharbo@gmail.com
> > > > > > > > >> > > > >wrote:
> > > > > > > > >> > > >
> > > > > > > > >> > > > > Glad you like it :)
> > > > > > > > >> > > > > Yes in fact the actual capture makes use of
> > > > > > > > View.capturePicture()
> > > > > > > > >> > which
> > > > > > > > >> > > > > actually grabs the entire document, then that can
> be
> > > > > clipped
> > > > > > > > down
> > > > > > > > >> to
> > > > > > > > >> > > the
> > > > > > > > >> > > > > size/location of a specific element.
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > -Aaron
> > > > > > > > >> > > > >
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <
> > > > > > > > >> mmocny@chromium.org>
> > > > > > > > >> > > > > wrote:
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > > Aaron,
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > I haven't even begun looking at your
> > implementation,
> > > > but
> > > > > > I'm
> > > > > > > > >> just
> > > > > > > > >> > > going
> > > > > > > > >> > > > > to
> > > > > > > > >> > > > > > say it: this is awesome!
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > First question: When Capturing a DOM element,
> can
> > > you
> > > > > > > capture
> > > > > > > > >> > 'body'
> > > > > > > > >> > > to
> > > > > > > > >> > > > > > grab it&children for a "full content
> screenshot",
> > or
> > > > > does
> > > > > > it
> > > > > > > > >> have
> > > > > > > > >> > to
> > > > > > > > >> > > > be a
> > > > > > > > >> > > > > > specific single element?
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > -Michal
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron
> > Charbonneau <
> > > > > > > > >> > > amcharbo@gmail.com
> > > > > > > > >> > > > > > >wrote:
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > > Greetings! My name is Aaron Charbonneau,
> happy
> > to
> > > > be
> > > > > a
> > > > > > > new
> > > > > > > > >> > member
> > > > > > > > >> > > of
> > > > > > > > >> > > > > the
> > > > > > > > >> > > > > > > mailing list!
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > I have been developing a sceenshot plugin for
> > > > Cordova
> > > > > to
> > > > > > > > help
> > > > > > > > >> > > > > facilitate
> > > > > > > > >> > > > > > > automation testing and debugging of Cordova
> > apps,
> > > > and
> > > > > I
> > > > > > > > would
> > > > > > > > >> > love
> > > > > > > > >> > > > some
> > > > > > > > >> > > > > > > feedback on it. Currently Cordova provides a
> > > bunch
> > > > of
> > > > > > > > native
> > > > > > > > >> > > > functions
> > > > > > > > >> > > > > > > that allow you to do some cool stuff, but not
> > much
> > > > > > > > >> functionality
> > > > > > > > >> > to
> > > > > > > > >> > > > > test
> > > > > > > > >> > > > > > > the apps that make use of them. Being able to
> > > take
> > > > a
> > > > > > > > capture
> > > > > > > > >> of
> > > > > > > > >> > > the
> > > > > > > > >> > > > > > screen
> > > > > > > > >> > > > > > > from within you app is a great way to automate
> > > > testing
> > > > > > or
> > > > > > > > get
> > > > > > > > >> > > > > additional
> > > > > > > > >> > > > > > > information for debugging an issue. Since
> there
> > > is
> > > > no
> > > > > > > > >> > > > > > > Javascript mechanism for taking screen
> captures
> > > the
> > > > > > > solution
> > > > > > > > >> > would
> > > > > > > > >> > > > have
> > > > > > > > >> > > > > > to
> > > > > > > > >> > > > > > > be native, which fits nicely into the "gap"
> that
> > > > > > > > >> Cordova/Phonegap
> > > > > > > > >> > > > > > bridges.
> > > > > > > > >> > > > > > > Any medium to large scale app can benefit
> > greatly
> > > > > from
> > > > > > > > >> > automation
> > > > > > > > >> > > > > > testing
> > > > > > > > >> > > > > > > and any app can benefit from an extra
> debugging
> > > > tool,
> > > > > > and
> > > > > > > > >> that is
> > > > > > > > >> > > > what
> > > > > > > > >> > > > > I
> > > > > > > > >> > > > > > > hope this screenshot plugin can help achieve.
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > Currently the plugin offers 2 functions:
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > Capture():
> > > > > > > > >> > > > > > > * Take a capture of the current view, write
> that
> > > > > capture
> > > > > > > to
> > > > > > > > a
> > > > > > > > >> > .png
> > > > > > > > >> > > > file
> > > > > > > > >> > > > > > > with the specified file name and sub directory
> > of
> > > > the
> > > > > > > sdcard
> > > > > > > > >> > > > (fallback
> > > > > > > > >> > > > > to
> > > > > > > > >> > > > > > > emulated sdcard in the case there isn't an
> > sdcard
> > > > > > mounted)
> > > > > > > > >> > > > > > > * Able to create a sub-screenshot with a
> > specified
> > > > > > > rectangle
> > > > > > > > >> in
> > > > > > > > >> > > order
> > > > > > > > >> > > > > to
> > > > > > > > >> > > > > > > block out ui elements that may be variable,
> and
> > > also
> > > > > > save
> > > > > > > > >> space.
> > > > > > > > >> > > > > > > * Can take captures of images/dom elements
> > > > (including
> > > > > > > > canvas)
> > > > > > > > >> > that
> > > > > > > > >> > > > are
> > > > > > > > >> > > > > > > lager than the actual screen size
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > CaptureAndCompare():
> > > > > > > > >> > > > > > > * All the functionality of Capture()
> > > > > > > > >> > > > > > > * Perform a comparison between the captured
> > image
> > > > and
> > > > > a
> > > > > > > > >> baseline
> > > > > > > > >> > > > image
> > > > > > > > >> > > > > > > located at the specified location in either
> the
> > > > assets
> > > > > > > > folder
> > > > > > > > >> or
> > > > > > > > >> > > the
> > > > > > > > >> > > > > > > sdcard.
> > > > > > > > >> > > > > > > * User can specify per color channel
> tolerances
> > as
> > > > > well
> > > > > > as
> > > > > > > > >> total
> > > > > > > > >> > > > pixel
> > > > > > > > >> > > > > > > tolerances to avoid false positives for the
> > > > inevitable
> > > > > > > > >> rendering
> > > > > > > > >> > > > > > > differences across devices.
> > > > > > > > >> > > > > > > * Optionally output a png file that contains
> the
> > > > > > > differences
> > > > > > > > >> > > between
> > > > > > > > >> > > > > the
> > > > > > > > >> > > > > > > actual and the baseline for debugging/triage
> > > > purposes,
> > > > > > two
> > > > > > > > >> modes:
> > > > > > > > >> > > > > binary
> > > > > > > > >> > > > > > > diff (all failing pixels appear as solid
> white)
> > or
> > > > the
> > > > > > > true
> > > > > > > > >> > > > differences
> > > > > > > > >> > > > > > > between pixels.
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > If you can spare some time, I would love it if
> > you
> > > > > could
> > > > > > > > take
> > > > > > > > >> a
> > > > > > > > >> > > look
> > > > > > > > >> > > > at
> > > > > > > > >> > > > > > the
> > > > > > > > >> > > > > > > api and parameters I have defined to make sure
> > > they
> > > > > > adhere
> > > > > > > > to
> > > > > > > > >> > > Cordova
> > > > > > > > >> > > > > > > plugin best practices. The most crucial part
> > > would
> > > > be
> > > > > > in
> > > > > > > > the
> > > > > > > > >> > > plugin
> > > > > > > > >> > > > > > itself
> > > > > > > > >> > > > > > > at ScreenCapture.java:
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > >
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > > > > > > >> > > > > > > and captureAndCompare() are the two exposed
> > > > functions)
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > I'm also interested to know you thoughts on
> it's
> > > > > usage.
> > > > > > I
> > > > > > > > >> > started
> > > > > > > > >> > > > with
> > > > > > > > >> > > > > > the
> > > > > > > > >> > > > > > > mobile-spec testing framework and put some
> quick
> > > > > > rendering
> > > > > > > > >> tests
> > > > > > > > >> > > into
> > > > > > > > >> > > > > the
> > > > > > > > >> > > > > > > Autotest section as a viable use case. In
> order
> > > to
> > > > > get
> > > > > > > the
> > > > > > > > >> > WebView
> > > > > > > > >> > > > > into
> > > > > > > > >> > > > > > > the state I wanted to capture, I had to
> > implement
> > > a
> > > > > wait
> > > > > > > > >> timer to
> > > > > > > > >> > > > allow
> > > > > > > > >> > > > > > the
> > > > > > > > >> > > > > > > view to update before taking the capture.
> Once
> > > the
> > > > > wait
> > > > > > > > >> function
> > > > > > > > >> > > > > > > completes, the capture can be taken and
> > everything
> > > > > from
> > > > > > > > there
> > > > > > > > >> is
> > > > > > > > >> > > > > callback
> > > > > > > > >> > > > > > > based. I use the Jasmine waitsFor()/runs()
> > blocks
> > > > to
> > > > > > make
> > > > > > > > the
> > > > > > > > >> > > tests
> > > > > > > > >> > > > > run
> > > > > > > > >> > > > > > > synchronously. For some validation testing I
> > > made a
> > > > > > > compare
> > > > > > > > >> > > function
> > > > > > > > >> > > > > in
> > > > > > > > >> > > > > > > Javascript to run against the native compare.
> > > Turns
> > > > > out
> > > > > > > > Java
> > > > > > > > >> > runs
> > > > > > > > >> > > > alot
> > > > > > > > >> > > > > > > faster than Javascript (surprise!) thus the
> > native
> > > > > > compare
> > > > > > > > is
> > > > > > > > >> > about
> > > > > > > > >> > > > > 2-5x
> > > > > > > > >> > > > > > > faster. All in all the process is fairly
> quick,
> > > the
> > > > > > full
> > > > > > > > >> > > > > > captureAndCompare
> > > > > > > > >> > > > > > > with 3 file io's takes about 233ms on a Nexus
> > One,
> > > > and
> > > > > > > > ~100ms
> > > > > > > > >> on
> > > > > > > > >> > a
> > > > > > > > >> > > > > Nexus
> > > > > > > > >> > > > > > 10
> > > > > > > > >> > > > > > > (256x256 image size).
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > Anyways here is the usage:
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > >
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > And here's the JS wrapper for the plugin
> calls:
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > >
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > Thanks for your time, and I look forward to
> ANY
> > > > > feedback
> > > > > > > > >> positive
> > > > > > > > >> > > or
> > > > > > > > >> > > > > > > negative.
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > Thanks,
> > > > > > > > >> > > > > > > Aaron
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > >
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Andrew Grieve <ag...@chromium.org>.
You could avoid the blocking queue if you dispatch the background thread's
runnable at the end of the UI thread's block.
I see you send a success callback as soon as the image is captured, but
before it is saved. This would improve throughput, but it might be even
more useful if you set the keepCallback flag on it and then send another
success response once you have the filename / comparison results. It's the
filename that the app is probably interested in.
On Fri, Mar 15, 2013 at 9:41 PM, Aaron Charbonneau <am...@gmail.com>wrote:
> Ah thanks for clarifying that.
>
> I'm not sure how I can alleviate the ui thread any more than I am
> currently. It is already doing the bare minimum amount of work on the
> there. Right now the only work done on the ui thread is
> view.capturePicture() which is the recommended thread for doing that work
> (otherwise you get a warning):
>
>
> mUIThreadDone = false;
> cordova.getActivity().runOnUiThread(new Runnable() {
> public void run() {
> CordovaWebView uiThreadView = webView;
> picture = uiThreadView.capturePicture();
> mUIThreadDone = true;
> }
> });
> //rest of the work is done on core thread
> while(!mUIThreadDone) {}
> //write the picture to a file
> ...
>
> I could do the file io and compares on a different thread than the
> core one, but for the major use case, the core thread needs to wait no
> matter what to ensure that the file is written and the comparison is
> complete, so I wouldn't expect much gain there.
>
>
> However, I've been experimenting with a more threaded version that
> does the file io and comparison on a background thread as you
> suggested. If you have time please check it out. While the code is
> not as clean, I think it addresses the issues you raised Andrew. For
> the major use case, performance is still the same as expected, but if
> we want pure throughput on captures there's a big gain from about 3
> captures per second (cps) to 10 cps on a Nexus 10.
>
>
> Here's the new plugin, with moar threading!
>
>
>
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
>
>
> Once again, thanks for all your feedback on this, I haven't had much
> experience with threading in java before now.
>
>
> -Aaron
>
>
>
>
>
> On Thu, Mar 14, 2013 at 8:10 AM, Andrew Grieve <ag...@chromium.org>
> wrote:
>
> > My suggestion was just to prevent the UI thread from locking up so that
> you
> > app doesn't appear to be frozen when the plugin is doing its thing.
> >
> >
> > On Wed, Mar 13, 2013 at 5:03 PM, Aaron Charbonneau <amcharbo@gmail.com
> > >wrote:
> >
> > > Yah that's how I have it working now :) No problem with that approach
> for
> > > working with Jasmine.
> > >
> > > I made reference to running them asynchronously to respond to Andrew
> > about
> > > returning immediately after the bits are captured, which in the case of
> > > using that with Jasmine would only see a benefit IF we could run
> multiple
> > > 'it' blocks at the same time.
> > >
> > > I think however Andrew was thinking of a different use case where we
> > aren't
> > > limited by a framework that requires tests to run serially, and results
> > can
> > > just come back as they are ready, and be evaluated all together at the
> > end,
> > > or not at all if you just want the captures and no evaluation.
> > >
> > > -Aaron
> > >
> > >
> > > On Wed, Mar 13, 2013 at 1:53 PM, Braden Shepherdson <
> braden@chromium.org
> > > >wrote:
> > >
> > > > As far as I know, you're correct: there's no way to move on the next
> > it()
> > > > block before the first one is done.
> > > >
> > > > What's the problem with just making the tests wait? Why does the next
> > > it()
> > > > block need to start early? You can make your call to Java, use
> waitFor
> > to
> > > > wait until the callback with the data is called, run your
> expectations,
> > > and
> > > > then move on to the next it() block. If the next it() block is using
> > the
> > > > result of this one, then they should be one it() block.
> > > >
> > > > Braden
> > > >
> > > >
> > > > On Wed, Mar 13, 2013 at 4:46 PM, Aaron Charbonneau <
> amcharbo@gmail.com
> > > > >wrote:
> > > >
> > > > > Hi Braden,
> > > > > Sorry maybe I am overlooking something. The way I understand it is
> > > that
> > > > a
> > > > > previous 'it' block needs to complete before the next one can run.
> > So
> > > I
> > > > > have various tests setup like this:
> > > > >
> > > > > it("renders something", function() {
> > > > > //draw something
> > > > > ....
> > > > > captureAndCompare(); //call the native function
> > > > > waitsFor(function() {
> > > > > return captureComplete; //set by the native function callback
> > > > > }, "capture never completed", 10000);
> > > > > runs(function() {
> > > > > expect(captureResult).toBe(0); //expects difference to be 0
> > > > > });
> > > > > });
> > > > > //next 'it' test
> > > > > ...
> > > > >
> > > > > The expect block needs to wait until there is a result returned
> from
> > > the
> > > > > native captureAndCompare call. Therefore we use a waitsFor/runs
> > block
> > > > > there.
> > > > >
> > > > > But say we wanted to allow Javascript to run immediately after
> > calling
> > > > that
> > > > > native function. The only thing you need to wait for is that Java
> > > > grabbed
> > > > > the bits, and the rest of the processing will be done 'at some
> point
> > in
> > > > the
> > > > > future.' So as Andrew suggested you would return as soon as the
> bits
> > > are
> > > > > captured, and allow JS to run again. The problem is that our
> expect
> > > > block
> > > > > needs the results, so that has to wait, the only other thing to do
> > > would
> > > > be
> > > > > to run the next 'it' block.
> > > > >
> > > > > This is the part I didn't think was possible, can the next 'it'
> block
> > > be
> > > > > run while the previous 'it' is still waiting to execute the
> 'expect'
> > > > block?
> > > > > Or is there another approach all together that would allow this?
> > > > >
> > > > > Thanks,
> > > > > Aaron
> > > > >
> > > > >
> > > > > On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <
> > > braden@chromium.org
> > > > > >wrote:
> > > > >
> > > > > > Jasmine does support asynchronous tests, and waiting an arbitrary
> > > > period
> > > > > or
> > > > > > until some condition is true. Why does that not work for these
> > tests?
> > > > > >
> > > > > > Braden
> > > > > >
> > > > > >
> > > > > > On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <
> > > amcharbo@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > Quick follow up question for the return values stuff:
> > > > > > > Does it make sense to have my functions called from execute
> > > > (capture()
> > > > > > and
> > > > > > > captureAndCompare()) return an error string if one is
> encountered
> > > and
> > > > > > pass
> > > > > > > that back in callbackContext.error, then have execute return
> > true?
> > > > > > >
> > > > > > > Or is there a preferred way to get errors from sub functions
> back
> > > > into
> > > > > > > Javascript?
> > > > > > >
> > > > > > > Thanks!
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <
> > > > > amcharbo@gmail.com
> > > > > > > >wrote:
> > > > > > >
> > > > > > > > Thanks Andrew these are great suggestions!
> > > > > > > >
> > > > > > > > About not needing the busy-wait in getScreenBits:
> > > > > > > > I'm not a fan of busy-waiting either.
> > > > > > > > Perhaps it's a side effect of Jasmine requiring tests to be
> run
> > > and
> > > > > > > > evaluated synchronously that I decided to use a busy-wait in
> > > > > > > getScreenBits.
> > > > > > > > If the Runnable calls CallbackContext.success/error once the
> > > bits
> > > > > are
> > > > > > > > captured (but not written to file) on the UIThread, the
> > > Javascript
> > > > > side
> > > > > > > > will proceed to run before it has an image url/compare
> result,
> > > most
> > > > > > > likely
> > > > > > > > causing that test to fail. As far as I know Jasmine doesn't
> > > allow
> > > > > for
> > > > > > > > kicking off a bunch of tests and then gather all the results
> at
> > > the
> > > > > end
> > > > > > > and
> > > > > > > > output pass/fails (each 'expect' function needs to be within
> an
> > > > 'it'
> > > > > > > block,
> > > > > > > > essentially that means one test must be evaluated before the
> > next
> > > > can
> > > > > > > > begin).
> > > > > > > >
> > > > > > > > I can see the usefulness of allowing the capture, file io,
> and
> > > > > > comparison
> > > > > > > > to be able to run asynchronously from the Javascript though.
> In
> > > the
> > > > > > case
> > > > > > > > where you have your own test framework that CAN kick off a
> > bunch
> > > of
> > > > > > tests
> > > > > > > > and just gather the results as they come in and do a batch
> > > > evaluation
> > > > > > at
> > > > > > > > the end, or in the case where you don't want to do any
> > evaluation
> > > > in
> > > > > > your
> > > > > > > > app and just want to get captures as fast as you can,
> > > asynchronous
> > > > > > could
> > > > > > > > increase performance there.
> > > > > > > >
> > > > > > > > I could try to implement it this way too, if you think
> that's a
> > > > worth
> > > > > > > > while use case, maybe just provide an 'asynchronous' flag in
> > > > > > > CaptureOptions?
> > > > > > > >
> > > > > > > > For doing the fileIO/compare using cordova.getThreadPool, is
> > > there
> > > > a
> > > > > > > > benefit to doing that if I still plan having the operation
> > block
> > > > the
> > > > > > > > Javascript? For asynchronous mode I see why that makes
> sense.
> > > > > > > >
> > > > > > > > For the return values, I'll make those changes right away!
> > > Thanks
> > > > > > again
> > > > > > > > for providing your insight, hopefully I understood it
> > correctly.
> > > > > > > >
> > > > > > > > -Aaron
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <
> > > > agrieve@chromium.org
> > > > > > > >wrote:
> > > > > > > >
> > > > > > > >> Had a glance at your code. Things you may want to address:
> > > > > > > >>
> > > > > > > >> while(!mUIThreadDone) {}
> > > > > > > >>
> > > > > > > >> You shouldn't need to busy-wait. Pass the CallbackContext to
> > the
> > > > > > > Runnable,
> > > > > > > >> and call success/error on it whenever you're done. No need
> to
> > > have
> > > > > the
> > > > > > > web
> > > > > > > >> thread wait for it. Even better would be to use a second
> > > Runnable
> > > > to
> > > > > > do
> > > > > > > >> the
> > > > > > > >> image compare / file IO on a background thread via
> > > > > > > cordova.getThreadPool()
> > > > > > > >>
> > > > > > > >> The return value to execute() should only be false if an
> > invalid
> > > > > > command
> > > > > > > >> was passed. It's not meant to indicate the success of the
> > > > operation.
> > > > > > You
> > > > > > > >> should be returning true for it.
> > > > > > > >>
> > > > > > > >> Instead of return "success"/"", it might simplify things a
> bit
> > > to
> > > > > > > return a
> > > > > > > >> boolean of true/false
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <
> > > > > lorin.beer.dev@gmail.com
> > > > > > > >> >wrote:
> > > > > > > >>
> > > > > > > >> > all the same, very impressive and faster then I expected.
> > > > > > > >> >
> > > > > > > >> > - Lorin
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <
> > > > > > > amcharbo@gmail.com
> > > > > > > >> > >wrote:
> > > > > > > >> >
> > > > > > > >> > > Thanks Lorin.
> > > > > > > >> > >
> > > > > > > >> > > I did a quick test for throughput on an animating
> canvas.
> > I
> > > > was
> > > > > > > able
> > > > > > > >> to
> > > > > > > >> > > grab 30 frames within a 5 seconds which equates to
> around
> > > > 166ms
> > > > > > per
> > > > > > > >> > capture
> > > > > > > >> > > on a Galaxy Nexus. Unfortunately not fast enough to
> > > automate
> > > > > > > >> something
> > > > > > > >> > > like video. As with any kind of capturing system, often
> > the
> > > > > > > overhead
> > > > > > > >> of
> > > > > > > >> > > copying bits around is enough to slow down the operation
> > of
> > > > the
> > > > > > app.
> > > > > > > >> > >
> > > > > > > >> > > -Aaron
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
> > > > > > > >> lorin.beer.dev@gmail.com
> > > > > > > >> > > >wrote:
> > > > > > > >> > >
> > > > > > > >> > > > Hey Aaron,
> > > > > > > >> > > >
> > > > > > > >> > > > very cool stuff, looking forward to checking it out!
> > > > > > > >> > > >
> > > > > > > >> > > > Question: any performance hit on the app by using
> > Capture?
> > > > > > What's
> > > > > > > >> the
> > > > > > > >> > > > expected throughput on images taken in this way
> (given a
> > > > > > > particular
> > > > > > > >> > > device,
> > > > > > > >> > > > say Galaxy Nexus)?
> > > > > > > >> > > >
> > > > > > > >> > > > - Lorin
> > > > > > > >> > > >
> > > > > > > >> > > >
> > > > > > > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <
> > > > > > > >> > amcharbo@gmail.com
> > > > > > > >> > > > >wrote:
> > > > > > > >> > > >
> > > > > > > >> > > > > Glad you like it :)
> > > > > > > >> > > > > Yes in fact the actual capture makes use of
> > > > > > > View.capturePicture()
> > > > > > > >> > which
> > > > > > > >> > > > > actually grabs the entire document, then that can be
> > > > clipped
> > > > > > > down
> > > > > > > >> to
> > > > > > > >> > > the
> > > > > > > >> > > > > size/location of a specific element.
> > > > > > > >> > > > >
> > > > > > > >> > > > > -Aaron
> > > > > > > >> > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <
> > > > > > > >> mmocny@chromium.org>
> > > > > > > >> > > > > wrote:
> > > > > > > >> > > > >
> > > > > > > >> > > > > > Aaron,
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > I haven't even begun looking at your
> implementation,
> > > but
> > > > > I'm
> > > > > > > >> just
> > > > > > > >> > > going
> > > > > > > >> > > > > to
> > > > > > > >> > > > > > say it: this is awesome!
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > First question: When Capturing a DOM element, can
> > you
> > > > > > capture
> > > > > > > >> > 'body'
> > > > > > > >> > > to
> > > > > > > >> > > > > > grab it&children for a "full content screenshot",
> or
> > > > does
> > > > > it
> > > > > > > >> have
> > > > > > > >> > to
> > > > > > > >> > > > be a
> > > > > > > >> > > > > > specific single element?
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > -Michal
> > > > > > > >> > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron
> Charbonneau <
> > > > > > > >> > > amcharbo@gmail.com
> > > > > > > >> > > > > > >wrote:
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > > Greetings! My name is Aaron Charbonneau, happy
> to
> > > be
> > > > a
> > > > > > new
> > > > > > > >> > member
> > > > > > > >> > > of
> > > > > > > >> > > > > the
> > > > > > > >> > > > > > > mailing list!
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > I have been developing a sceenshot plugin for
> > > Cordova
> > > > to
> > > > > > > help
> > > > > > > >> > > > > facilitate
> > > > > > > >> > > > > > > automation testing and debugging of Cordova
> apps,
> > > and
> > > > I
> > > > > > > would
> > > > > > > >> > love
> > > > > > > >> > > > some
> > > > > > > >> > > > > > > feedback on it. Currently Cordova provides a
> > bunch
> > > of
> > > > > > > native
> > > > > > > >> > > > functions
> > > > > > > >> > > > > > > that allow you to do some cool stuff, but not
> much
> > > > > > > >> functionality
> > > > > > > >> > to
> > > > > > > >> > > > > test
> > > > > > > >> > > > > > > the apps that make use of them. Being able to
> > take
> > > a
> > > > > > > capture
> > > > > > > >> of
> > > > > > > >> > > the
> > > > > > > >> > > > > > screen
> > > > > > > >> > > > > > > from within you app is a great way to automate
> > > testing
> > > > > or
> > > > > > > get
> > > > > > > >> > > > > additional
> > > > > > > >> > > > > > > information for debugging an issue. Since there
> > is
> > > no
> > > > > > > >> > > > > > > Javascript mechanism for taking screen captures
> > the
> > > > > > solution
> > > > > > > >> > would
> > > > > > > >> > > > have
> > > > > > > >> > > > > > to
> > > > > > > >> > > > > > > be native, which fits nicely into the "gap" that
> > > > > > > >> Cordova/Phonegap
> > > > > > > >> > > > > > bridges.
> > > > > > > >> > > > > > > Any medium to large scale app can benefit
> greatly
> > > > from
> > > > > > > >> > automation
> > > > > > > >> > > > > > testing
> > > > > > > >> > > > > > > and any app can benefit from an extra debugging
> > > tool,
> > > > > and
> > > > > > > >> that is
> > > > > > > >> > > > what
> > > > > > > >> > > > > I
> > > > > > > >> > > > > > > hope this screenshot plugin can help achieve.
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > Currently the plugin offers 2 functions:
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > Capture():
> > > > > > > >> > > > > > > * Take a capture of the current view, write that
> > > > capture
> > > > > > to
> > > > > > > a
> > > > > > > >> > .png
> > > > > > > >> > > > file
> > > > > > > >> > > > > > > with the specified file name and sub directory
> of
> > > the
> > > > > > sdcard
> > > > > > > >> > > > (fallback
> > > > > > > >> > > > > to
> > > > > > > >> > > > > > > emulated sdcard in the case there isn't an
> sdcard
> > > > > mounted)
> > > > > > > >> > > > > > > * Able to create a sub-screenshot with a
> specified
> > > > > > rectangle
> > > > > > > >> in
> > > > > > > >> > > order
> > > > > > > >> > > > > to
> > > > > > > >> > > > > > > block out ui elements that may be variable, and
> > also
> > > > > save
> > > > > > > >> space.
> > > > > > > >> > > > > > > * Can take captures of images/dom elements
> > > (including
> > > > > > > canvas)
> > > > > > > >> > that
> > > > > > > >> > > > are
> > > > > > > >> > > > > > > lager than the actual screen size
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > CaptureAndCompare():
> > > > > > > >> > > > > > > * All the functionality of Capture()
> > > > > > > >> > > > > > > * Perform a comparison between the captured
> image
> > > and
> > > > a
> > > > > > > >> baseline
> > > > > > > >> > > > image
> > > > > > > >> > > > > > > located at the specified location in either the
> > > assets
> > > > > > > folder
> > > > > > > >> or
> > > > > > > >> > > the
> > > > > > > >> > > > > > > sdcard.
> > > > > > > >> > > > > > > * User can specify per color channel tolerances
> as
> > > > well
> > > > > as
> > > > > > > >> total
> > > > > > > >> > > > pixel
> > > > > > > >> > > > > > > tolerances to avoid false positives for the
> > > inevitable
> > > > > > > >> rendering
> > > > > > > >> > > > > > > differences across devices.
> > > > > > > >> > > > > > > * Optionally output a png file that contains the
> > > > > > differences
> > > > > > > >> > > between
> > > > > > > >> > > > > the
> > > > > > > >> > > > > > > actual and the baseline for debugging/triage
> > > purposes,
> > > > > two
> > > > > > > >> modes:
> > > > > > > >> > > > > binary
> > > > > > > >> > > > > > > diff (all failing pixels appear as solid white)
> or
> > > the
> > > > > > true
> > > > > > > >> > > > differences
> > > > > > > >> > > > > > > between pixels.
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > If you can spare some time, I would love it if
> you
> > > > could
> > > > > > > take
> > > > > > > >> a
> > > > > > > >> > > look
> > > > > > > >> > > > at
> > > > > > > >> > > > > > the
> > > > > > > >> > > > > > > api and parameters I have defined to make sure
> > they
> > > > > adhere
> > > > > > > to
> > > > > > > >> > > Cordova
> > > > > > > >> > > > > > > plugin best practices. The most crucial part
> > would
> > > be
> > > > > in
> > > > > > > the
> > > > > > > >> > > plugin
> > > > > > > >> > > > > > itself
> > > > > > > >> > > > > > > at ScreenCapture.java:
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > > > > > >> > > > > > > and captureAndCompare() are the two exposed
> > > functions)
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > I'm also interested to know you thoughts on it's
> > > > usage.
> > > > > I
> > > > > > > >> > started
> > > > > > > >> > > > with
> > > > > > > >> > > > > > the
> > > > > > > >> > > > > > > mobile-spec testing framework and put some quick
> > > > > rendering
> > > > > > > >> tests
> > > > > > > >> > > into
> > > > > > > >> > > > > the
> > > > > > > >> > > > > > > Autotest section as a viable use case. In order
> > to
> > > > get
> > > > > > the
> > > > > > > >> > WebView
> > > > > > > >> > > > > into
> > > > > > > >> > > > > > > the state I wanted to capture, I had to
> implement
> > a
> > > > wait
> > > > > > > >> timer to
> > > > > > > >> > > > allow
> > > > > > > >> > > > > > the
> > > > > > > >> > > > > > > view to update before taking the capture. Once
> > the
> > > > wait
> > > > > > > >> function
> > > > > > > >> > > > > > > completes, the capture can be taken and
> everything
> > > > from
> > > > > > > there
> > > > > > > >> is
> > > > > > > >> > > > > callback
> > > > > > > >> > > > > > > based. I use the Jasmine waitsFor()/runs()
> blocks
> > > to
> > > > > make
> > > > > > > the
> > > > > > > >> > > tests
> > > > > > > >> > > > > run
> > > > > > > >> > > > > > > synchronously. For some validation testing I
> > made a
> > > > > > compare
> > > > > > > >> > > function
> > > > > > > >> > > > > in
> > > > > > > >> > > > > > > Javascript to run against the native compare.
> > Turns
> > > > out
> > > > > > > Java
> > > > > > > >> > runs
> > > > > > > >> > > > alot
> > > > > > > >> > > > > > > faster than Javascript (surprise!) thus the
> native
> > > > > compare
> > > > > > > is
> > > > > > > >> > about
> > > > > > > >> > > > > 2-5x
> > > > > > > >> > > > > > > faster. All in all the process is fairly quick,
> > the
> > > > > full
> > > > > > > >> > > > > > captureAndCompare
> > > > > > > >> > > > > > > with 3 file io's takes about 233ms on a Nexus
> One,
> > > and
> > > > > > > ~100ms
> > > > > > > >> on
> > > > > > > >> > a
> > > > > > > >> > > > > Nexus
> > > > > > > >> > > > > > 10
> > > > > > > >> > > > > > > (256x256 image size).
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > Anyways here is the usage:
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > And here's the JS wrapper for the plugin calls:
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > Thanks for your time, and I look forward to ANY
> > > > feedback
> > > > > > > >> positive
> > > > > > > >> > > or
> > > > > > > >> > > > > > > negative.
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > Thanks,
> > > > > > > >> > > > > > > Aaron
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Aaron Charbonneau <am...@gmail.com>.
Ah thanks for clarifying that.
I'm not sure how I can alleviate the ui thread any more than I am
currently. It is already doing the bare minimum amount of work on the
there. Right now the only work done on the ui thread is
view.capturePicture() which is the recommended thread for doing that work
(otherwise you get a warning):
mUIThreadDone = false;
cordova.getActivity().runOnUiThread(new Runnable() {
public void run() {
CordovaWebView uiThreadView = webView;
picture = uiThreadView.capturePicture();
mUIThreadDone = true;
}
});
//rest of the work is done on core thread
while(!mUIThreadDone) {}
//write the picture to a file
...
I could do the file io and compares on a different thread than the
core one, but for the major use case, the core thread needs to wait no
matter what to ensure that the file is written and the comparison is
complete, so I wouldn't expect much gain there.
However, I've been experimenting with a more threaded version that
does the file io and comparison on a background thread as you
suggested. If you have time please check it out. While the code is
not as clean, I think it addresses the issues you raised Andrew. For
the major use case, performance is still the same as expected, but if
we want pure throughput on captures there's a big gain from about 3
captures per second (cps) to 10 cps on a Nexus 10.
Here's the new plugin, with moar threading!
https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
Once again, thanks for all your feedback on this, I haven't had much
experience with threading in java before now.
-Aaron
On Thu, Mar 14, 2013 at 8:10 AM, Andrew Grieve <ag...@chromium.org> wrote:
> My suggestion was just to prevent the UI thread from locking up so that you
> app doesn't appear to be frozen when the plugin is doing its thing.
>
>
> On Wed, Mar 13, 2013 at 5:03 PM, Aaron Charbonneau <amcharbo@gmail.com
> >wrote:
>
> > Yah that's how I have it working now :) No problem with that approach for
> > working with Jasmine.
> >
> > I made reference to running them asynchronously to respond to Andrew
> about
> > returning immediately after the bits are captured, which in the case of
> > using that with Jasmine would only see a benefit IF we could run multiple
> > 'it' blocks at the same time.
> >
> > I think however Andrew was thinking of a different use case where we
> aren't
> > limited by a framework that requires tests to run serially, and results
> can
> > just come back as they are ready, and be evaluated all together at the
> end,
> > or not at all if you just want the captures and no evaluation.
> >
> > -Aaron
> >
> >
> > On Wed, Mar 13, 2013 at 1:53 PM, Braden Shepherdson <braden@chromium.org
> > >wrote:
> >
> > > As far as I know, you're correct: there's no way to move on the next
> it()
> > > block before the first one is done.
> > >
> > > What's the problem with just making the tests wait? Why does the next
> > it()
> > > block need to start early? You can make your call to Java, use waitFor
> to
> > > wait until the callback with the data is called, run your expectations,
> > and
> > > then move on to the next it() block. If the next it() block is using
> the
> > > result of this one, then they should be one it() block.
> > >
> > > Braden
> > >
> > >
> > > On Wed, Mar 13, 2013 at 4:46 PM, Aaron Charbonneau <amcharbo@gmail.com
> > > >wrote:
> > >
> > > > Hi Braden,
> > > > Sorry maybe I am overlooking something. The way I understand it is
> > that
> > > a
> > > > previous 'it' block needs to complete before the next one can run.
> So
> > I
> > > > have various tests setup like this:
> > > >
> > > > it("renders something", function() {
> > > > //draw something
> > > > ....
> > > > captureAndCompare(); //call the native function
> > > > waitsFor(function() {
> > > > return captureComplete; //set by the native function callback
> > > > }, "capture never completed", 10000);
> > > > runs(function() {
> > > > expect(captureResult).toBe(0); //expects difference to be 0
> > > > });
> > > > });
> > > > //next 'it' test
> > > > ...
> > > >
> > > > The expect block needs to wait until there is a result returned from
> > the
> > > > native captureAndCompare call. Therefore we use a waitsFor/runs
> block
> > > > there.
> > > >
> > > > But say we wanted to allow Javascript to run immediately after
> calling
> > > that
> > > > native function. The only thing you need to wait for is that Java
> > > grabbed
> > > > the bits, and the rest of the processing will be done 'at some point
> in
> > > the
> > > > future.' So as Andrew suggested you would return as soon as the bits
> > are
> > > > captured, and allow JS to run again. The problem is that our expect
> > > block
> > > > needs the results, so that has to wait, the only other thing to do
> > would
> > > be
> > > > to run the next 'it' block.
> > > >
> > > > This is the part I didn't think was possible, can the next 'it' block
> > be
> > > > run while the previous 'it' is still waiting to execute the 'expect'
> > > block?
> > > > Or is there another approach all together that would allow this?
> > > >
> > > > Thanks,
> > > > Aaron
> > > >
> > > >
> > > > On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <
> > braden@chromium.org
> > > > >wrote:
> > > >
> > > > > Jasmine does support asynchronous tests, and waiting an arbitrary
> > > period
> > > > or
> > > > > until some condition is true. Why does that not work for these
> tests?
> > > > >
> > > > > Braden
> > > > >
> > > > >
> > > > > On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <
> > amcharbo@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Quick follow up question for the return values stuff:
> > > > > > Does it make sense to have my functions called from execute
> > > (capture()
> > > > > and
> > > > > > captureAndCompare()) return an error string if one is encountered
> > and
> > > > > pass
> > > > > > that back in callbackContext.error, then have execute return
> true?
> > > > > >
> > > > > > Or is there a preferred way to get errors from sub functions back
> > > into
> > > > > > Javascript?
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > >
> > > > > > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <
> > > > amcharbo@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > Thanks Andrew these are great suggestions!
> > > > > > >
> > > > > > > About not needing the busy-wait in getScreenBits:
> > > > > > > I'm not a fan of busy-waiting either.
> > > > > > > Perhaps it's a side effect of Jasmine requiring tests to be run
> > and
> > > > > > > evaluated synchronously that I decided to use a busy-wait in
> > > > > > getScreenBits.
> > > > > > > If the Runnable calls CallbackContext.success/error once the
> > bits
> > > > are
> > > > > > > captured (but not written to file) on the UIThread, the
> > Javascript
> > > > side
> > > > > > > will proceed to run before it has an image url/compare result,
> > most
> > > > > > likely
> > > > > > > causing that test to fail. As far as I know Jasmine doesn't
> > allow
> > > > for
> > > > > > > kicking off a bunch of tests and then gather all the results at
> > the
> > > > end
> > > > > > and
> > > > > > > output pass/fails (each 'expect' function needs to be within an
> > > 'it'
> > > > > > block,
> > > > > > > essentially that means one test must be evaluated before the
> next
> > > can
> > > > > > > begin).
> > > > > > >
> > > > > > > I can see the usefulness of allowing the capture, file io, and
> > > > > comparison
> > > > > > > to be able to run asynchronously from the Javascript though. In
> > the
> > > > > case
> > > > > > > where you have your own test framework that CAN kick off a
> bunch
> > of
> > > > > tests
> > > > > > > and just gather the results as they come in and do a batch
> > > evaluation
> > > > > at
> > > > > > > the end, or in the case where you don't want to do any
> evaluation
> > > in
> > > > > your
> > > > > > > app and just want to get captures as fast as you can,
> > asynchronous
> > > > > could
> > > > > > > increase performance there.
> > > > > > >
> > > > > > > I could try to implement it this way too, if you think that's a
> > > worth
> > > > > > > while use case, maybe just provide an 'asynchronous' flag in
> > > > > > CaptureOptions?
> > > > > > >
> > > > > > > For doing the fileIO/compare using cordova.getThreadPool, is
> > there
> > > a
> > > > > > > benefit to doing that if I still plan having the operation
> block
> > > the
> > > > > > > Javascript? For asynchronous mode I see why that makes sense.
> > > > > > >
> > > > > > > For the return values, I'll make those changes right away!
> > Thanks
> > > > > again
> > > > > > > for providing your insight, hopefully I understood it
> correctly.
> > > > > > >
> > > > > > > -Aaron
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <
> > > agrieve@chromium.org
> > > > > > >wrote:
> > > > > > >
> > > > > > >> Had a glance at your code. Things you may want to address:
> > > > > > >>
> > > > > > >> while(!mUIThreadDone) {}
> > > > > > >>
> > > > > > >> You shouldn't need to busy-wait. Pass the CallbackContext to
> the
> > > > > > Runnable,
> > > > > > >> and call success/error on it whenever you're done. No need to
> > have
> > > > the
> > > > > > web
> > > > > > >> thread wait for it. Even better would be to use a second
> > Runnable
> > > to
> > > > > do
> > > > > > >> the
> > > > > > >> image compare / file IO on a background thread via
> > > > > > cordova.getThreadPool()
> > > > > > >>
> > > > > > >> The return value to execute() should only be false if an
> invalid
> > > > > command
> > > > > > >> was passed. It's not meant to indicate the success of the
> > > operation.
> > > > > You
> > > > > > >> should be returning true for it.
> > > > > > >>
> > > > > > >> Instead of return "success"/"", it might simplify things a bit
> > to
> > > > > > return a
> > > > > > >> boolean of true/false
> > > > > > >>
> > > > > > >>
> > > > > > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <
> > > > lorin.beer.dev@gmail.com
> > > > > > >> >wrote:
> > > > > > >>
> > > > > > >> > all the same, very impressive and faster then I expected.
> > > > > > >> >
> > > > > > >> > - Lorin
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <
> > > > > > amcharbo@gmail.com
> > > > > > >> > >wrote:
> > > > > > >> >
> > > > > > >> > > Thanks Lorin.
> > > > > > >> > >
> > > > > > >> > > I did a quick test for throughput on an animating canvas.
> I
> > > was
> > > > > > able
> > > > > > >> to
> > > > > > >> > > grab 30 frames within a 5 seconds which equates to around
> > > 166ms
> > > > > per
> > > > > > >> > capture
> > > > > > >> > > on a Galaxy Nexus. Unfortunately not fast enough to
> > automate
> > > > > > >> something
> > > > > > >> > > like video. As with any kind of capturing system, often
> the
> > > > > > overhead
> > > > > > >> of
> > > > > > >> > > copying bits around is enough to slow down the operation
> of
> > > the
> > > > > app.
> > > > > > >> > >
> > > > > > >> > > -Aaron
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
> > > > > > >> lorin.beer.dev@gmail.com
> > > > > > >> > > >wrote:
> > > > > > >> > >
> > > > > > >> > > > Hey Aaron,
> > > > > > >> > > >
> > > > > > >> > > > very cool stuff, looking forward to checking it out!
> > > > > > >> > > >
> > > > > > >> > > > Question: any performance hit on the app by using
> Capture?
> > > > > What's
> > > > > > >> the
> > > > > > >> > > > expected throughput on images taken in this way (given a
> > > > > > particular
> > > > > > >> > > device,
> > > > > > >> > > > say Galaxy Nexus)?
> > > > > > >> > > >
> > > > > > >> > > > - Lorin
> > > > > > >> > > >
> > > > > > >> > > >
> > > > > > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <
> > > > > > >> > amcharbo@gmail.com
> > > > > > >> > > > >wrote:
> > > > > > >> > > >
> > > > > > >> > > > > Glad you like it :)
> > > > > > >> > > > > Yes in fact the actual capture makes use of
> > > > > > View.capturePicture()
> > > > > > >> > which
> > > > > > >> > > > > actually grabs the entire document, then that can be
> > > clipped
> > > > > > down
> > > > > > >> to
> > > > > > >> > > the
> > > > > > >> > > > > size/location of a specific element.
> > > > > > >> > > > >
> > > > > > >> > > > > -Aaron
> > > > > > >> > > > >
> > > > > > >> > > > >
> > > > > > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <
> > > > > > >> mmocny@chromium.org>
> > > > > > >> > > > > wrote:
> > > > > > >> > > > >
> > > > > > >> > > > > > Aaron,
> > > > > > >> > > > > >
> > > > > > >> > > > > > I haven't even begun looking at your implementation,
> > but
> > > > I'm
> > > > > > >> just
> > > > > > >> > > going
> > > > > > >> > > > > to
> > > > > > >> > > > > > say it: this is awesome!
> > > > > > >> > > > > >
> > > > > > >> > > > > > First question: When Capturing a DOM element, can
> you
> > > > > capture
> > > > > > >> > 'body'
> > > > > > >> > > to
> > > > > > >> > > > > > grab it&children for a "full content screenshot", or
> > > does
> > > > it
> > > > > > >> have
> > > > > > >> > to
> > > > > > >> > > > be a
> > > > > > >> > > > > > specific single element?
> > > > > > >> > > > > >
> > > > > > >> > > > > > -Michal
> > > > > > >> > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <
> > > > > > >> > > amcharbo@gmail.com
> > > > > > >> > > > > > >wrote:
> > > > > > >> > > > > >
> > > > > > >> > > > > > > Greetings! My name is Aaron Charbonneau, happy to
> > be
> > > a
> > > > > new
> > > > > > >> > member
> > > > > > >> > > of
> > > > > > >> > > > > the
> > > > > > >> > > > > > > mailing list!
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > I have been developing a sceenshot plugin for
> > Cordova
> > > to
> > > > > > help
> > > > > > >> > > > > facilitate
> > > > > > >> > > > > > > automation testing and debugging of Cordova apps,
> > and
> > > I
> > > > > > would
> > > > > > >> > love
> > > > > > >> > > > some
> > > > > > >> > > > > > > feedback on it. Currently Cordova provides a
> bunch
> > of
> > > > > > native
> > > > > > >> > > > functions
> > > > > > >> > > > > > > that allow you to do some cool stuff, but not much
> > > > > > >> functionality
> > > > > > >> > to
> > > > > > >> > > > > test
> > > > > > >> > > > > > > the apps that make use of them. Being able to
> take
> > a
> > > > > > capture
> > > > > > >> of
> > > > > > >> > > the
> > > > > > >> > > > > > screen
> > > > > > >> > > > > > > from within you app is a great way to automate
> > testing
> > > > or
> > > > > > get
> > > > > > >> > > > > additional
> > > > > > >> > > > > > > information for debugging an issue. Since there
> is
> > no
> > > > > > >> > > > > > > Javascript mechanism for taking screen captures
> the
> > > > > solution
> > > > > > >> > would
> > > > > > >> > > > have
> > > > > > >> > > > > > to
> > > > > > >> > > > > > > be native, which fits nicely into the "gap" that
> > > > > > >> Cordova/Phonegap
> > > > > > >> > > > > > bridges.
> > > > > > >> > > > > > > Any medium to large scale app can benefit greatly
> > > from
> > > > > > >> > automation
> > > > > > >> > > > > > testing
> > > > > > >> > > > > > > and any app can benefit from an extra debugging
> > tool,
> > > > and
> > > > > > >> that is
> > > > > > >> > > > what
> > > > > > >> > > > > I
> > > > > > >> > > > > > > hope this screenshot plugin can help achieve.
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Currently the plugin offers 2 functions:
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Capture():
> > > > > > >> > > > > > > * Take a capture of the current view, write that
> > > capture
> > > > > to
> > > > > > a
> > > > > > >> > .png
> > > > > > >> > > > file
> > > > > > >> > > > > > > with the specified file name and sub directory of
> > the
> > > > > sdcard
> > > > > > >> > > > (fallback
> > > > > > >> > > > > to
> > > > > > >> > > > > > > emulated sdcard in the case there isn't an sdcard
> > > > mounted)
> > > > > > >> > > > > > > * Able to create a sub-screenshot with a specified
> > > > > rectangle
> > > > > > >> in
> > > > > > >> > > order
> > > > > > >> > > > > to
> > > > > > >> > > > > > > block out ui elements that may be variable, and
> also
> > > > save
> > > > > > >> space.
> > > > > > >> > > > > > > * Can take captures of images/dom elements
> > (including
> > > > > > canvas)
> > > > > > >> > that
> > > > > > >> > > > are
> > > > > > >> > > > > > > lager than the actual screen size
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > CaptureAndCompare():
> > > > > > >> > > > > > > * All the functionality of Capture()
> > > > > > >> > > > > > > * Perform a comparison between the captured image
> > and
> > > a
> > > > > > >> baseline
> > > > > > >> > > > image
> > > > > > >> > > > > > > located at the specified location in either the
> > assets
> > > > > > folder
> > > > > > >> or
> > > > > > >> > > the
> > > > > > >> > > > > > > sdcard.
> > > > > > >> > > > > > > * User can specify per color channel tolerances as
> > > well
> > > > as
> > > > > > >> total
> > > > > > >> > > > pixel
> > > > > > >> > > > > > > tolerances to avoid false positives for the
> > inevitable
> > > > > > >> rendering
> > > > > > >> > > > > > > differences across devices.
> > > > > > >> > > > > > > * Optionally output a png file that contains the
> > > > > differences
> > > > > > >> > > between
> > > > > > >> > > > > the
> > > > > > >> > > > > > > actual and the baseline for debugging/triage
> > purposes,
> > > > two
> > > > > > >> modes:
> > > > > > >> > > > > binary
> > > > > > >> > > > > > > diff (all failing pixels appear as solid white) or
> > the
> > > > > true
> > > > > > >> > > > differences
> > > > > > >> > > > > > > between pixels.
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > If you can spare some time, I would love it if you
> > > could
> > > > > > take
> > > > > > >> a
> > > > > > >> > > look
> > > > > > >> > > > at
> > > > > > >> > > > > > the
> > > > > > >> > > > > > > api and parameters I have defined to make sure
> they
> > > > adhere
> > > > > > to
> > > > > > >> > > Cordova
> > > > > > >> > > > > > > plugin best practices. The most crucial part
> would
> > be
> > > > in
> > > > > > the
> > > > > > >> > > plugin
> > > > > > >> > > > > > itself
> > > > > > >> > > > > > > at ScreenCapture.java:
> > > > > > >> > > > > > >
> > > > > > >> > > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > > > > >> > > > > > > and captureAndCompare() are the two exposed
> > functions)
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > I'm also interested to know you thoughts on it's
> > > usage.
> > > > I
> > > > > > >> > started
> > > > > > >> > > > with
> > > > > > >> > > > > > the
> > > > > > >> > > > > > > mobile-spec testing framework and put some quick
> > > > rendering
> > > > > > >> tests
> > > > > > >> > > into
> > > > > > >> > > > > the
> > > > > > >> > > > > > > Autotest section as a viable use case. In order
> to
> > > get
> > > > > the
> > > > > > >> > WebView
> > > > > > >> > > > > into
> > > > > > >> > > > > > > the state I wanted to capture, I had to implement
> a
> > > wait
> > > > > > >> timer to
> > > > > > >> > > > allow
> > > > > > >> > > > > > the
> > > > > > >> > > > > > > view to update before taking the capture. Once
> the
> > > wait
> > > > > > >> function
> > > > > > >> > > > > > > completes, the capture can be taken and everything
> > > from
> > > > > > there
> > > > > > >> is
> > > > > > >> > > > > callback
> > > > > > >> > > > > > > based. I use the Jasmine waitsFor()/runs() blocks
> > to
> > > > make
> > > > > > the
> > > > > > >> > > tests
> > > > > > >> > > > > run
> > > > > > >> > > > > > > synchronously. For some validation testing I
> made a
> > > > > compare
> > > > > > >> > > function
> > > > > > >> > > > > in
> > > > > > >> > > > > > > Javascript to run against the native compare.
> Turns
> > > out
> > > > > > Java
> > > > > > >> > runs
> > > > > > >> > > > alot
> > > > > > >> > > > > > > faster than Javascript (surprise!) thus the native
> > > > compare
> > > > > > is
> > > > > > >> > about
> > > > > > >> > > > > 2-5x
> > > > > > >> > > > > > > faster. All in all the process is fairly quick,
> the
> > > > full
> > > > > > >> > > > > > captureAndCompare
> > > > > > >> > > > > > > with 3 file io's takes about 233ms on a Nexus One,
> > and
> > > > > > ~100ms
> > > > > > >> on
> > > > > > >> > a
> > > > > > >> > > > > Nexus
> > > > > > >> > > > > > 10
> > > > > > >> > > > > > > (256x256 image size).
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Anyways here is the usage:
> > > > > > >> > > > > > >
> > > > > > >> > > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > And here's the JS wrapper for the plugin calls:
> > > > > > >> > > > > > >
> > > > > > >> > > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Thanks for your time, and I look forward to ANY
> > > feedback
> > > > > > >> positive
> > > > > > >> > > or
> > > > > > >> > > > > > > negative.
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Thanks,
> > > > > > >> > > > > > > Aaron
> > > > > > >> > > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Andrew Grieve <ag...@chromium.org>.
My suggestion was just to prevent the UI thread from locking up so that you
app doesn't appear to be frozen when the plugin is doing its thing.
On Wed, Mar 13, 2013 at 5:03 PM, Aaron Charbonneau <am...@gmail.com>wrote:
> Yah that's how I have it working now :) No problem with that approach for
> working with Jasmine.
>
> I made reference to running them asynchronously to respond to Andrew about
> returning immediately after the bits are captured, which in the case of
> using that with Jasmine would only see a benefit IF we could run multiple
> 'it' blocks at the same time.
>
> I think however Andrew was thinking of a different use case where we aren't
> limited by a framework that requires tests to run serially, and results can
> just come back as they are ready, and be evaluated all together at the end,
> or not at all if you just want the captures and no evaluation.
>
> -Aaron
>
>
> On Wed, Mar 13, 2013 at 1:53 PM, Braden Shepherdson <braden@chromium.org
> >wrote:
>
> > As far as I know, you're correct: there's no way to move on the next it()
> > block before the first one is done.
> >
> > What's the problem with just making the tests wait? Why does the next
> it()
> > block need to start early? You can make your call to Java, use waitFor to
> > wait until the callback with the data is called, run your expectations,
> and
> > then move on to the next it() block. If the next it() block is using the
> > result of this one, then they should be one it() block.
> >
> > Braden
> >
> >
> > On Wed, Mar 13, 2013 at 4:46 PM, Aaron Charbonneau <amcharbo@gmail.com
> > >wrote:
> >
> > > Hi Braden,
> > > Sorry maybe I am overlooking something. The way I understand it is
> that
> > a
> > > previous 'it' block needs to complete before the next one can run. So
> I
> > > have various tests setup like this:
> > >
> > > it("renders something", function() {
> > > //draw something
> > > ....
> > > captureAndCompare(); //call the native function
> > > waitsFor(function() {
> > > return captureComplete; //set by the native function callback
> > > }, "capture never completed", 10000);
> > > runs(function() {
> > > expect(captureResult).toBe(0); //expects difference to be 0
> > > });
> > > });
> > > //next 'it' test
> > > ...
> > >
> > > The expect block needs to wait until there is a result returned from
> the
> > > native captureAndCompare call. Therefore we use a waitsFor/runs block
> > > there.
> > >
> > > But say we wanted to allow Javascript to run immediately after calling
> > that
> > > native function. The only thing you need to wait for is that Java
> > grabbed
> > > the bits, and the rest of the processing will be done 'at some point in
> > the
> > > future.' So as Andrew suggested you would return as soon as the bits
> are
> > > captured, and allow JS to run again. The problem is that our expect
> > block
> > > needs the results, so that has to wait, the only other thing to do
> would
> > be
> > > to run the next 'it' block.
> > >
> > > This is the part I didn't think was possible, can the next 'it' block
> be
> > > run while the previous 'it' is still waiting to execute the 'expect'
> > block?
> > > Or is there another approach all together that would allow this?
> > >
> > > Thanks,
> > > Aaron
> > >
> > >
> > > On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <
> braden@chromium.org
> > > >wrote:
> > >
> > > > Jasmine does support asynchronous tests, and waiting an arbitrary
> > period
> > > or
> > > > until some condition is true. Why does that not work for these tests?
> > > >
> > > > Braden
> > > >
> > > >
> > > > On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <
> amcharbo@gmail.com
> > > > >wrote:
> > > >
> > > > > Quick follow up question for the return values stuff:
> > > > > Does it make sense to have my functions called from execute
> > (capture()
> > > > and
> > > > > captureAndCompare()) return an error string if one is encountered
> and
> > > > pass
> > > > > that back in callbackContext.error, then have execute return true?
> > > > >
> > > > > Or is there a preferred way to get errors from sub functions back
> > into
> > > > > Javascript?
> > > > >
> > > > > Thanks!
> > > > >
> > > > >
> > > > > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <
> > > amcharbo@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Thanks Andrew these are great suggestions!
> > > > > >
> > > > > > About not needing the busy-wait in getScreenBits:
> > > > > > I'm not a fan of busy-waiting either.
> > > > > > Perhaps it's a side effect of Jasmine requiring tests to be run
> and
> > > > > > evaluated synchronously that I decided to use a busy-wait in
> > > > > getScreenBits.
> > > > > > If the Runnable calls CallbackContext.success/error once the
> bits
> > > are
> > > > > > captured (but not written to file) on the UIThread, the
> Javascript
> > > side
> > > > > > will proceed to run before it has an image url/compare result,
> most
> > > > > likely
> > > > > > causing that test to fail. As far as I know Jasmine doesn't
> allow
> > > for
> > > > > > kicking off a bunch of tests and then gather all the results at
> the
> > > end
> > > > > and
> > > > > > output pass/fails (each 'expect' function needs to be within an
> > 'it'
> > > > > block,
> > > > > > essentially that means one test must be evaluated before the next
> > can
> > > > > > begin).
> > > > > >
> > > > > > I can see the usefulness of allowing the capture, file io, and
> > > > comparison
> > > > > > to be able to run asynchronously from the Javascript though. In
> the
> > > > case
> > > > > > where you have your own test framework that CAN kick off a bunch
> of
> > > > tests
> > > > > > and just gather the results as they come in and do a batch
> > evaluation
> > > > at
> > > > > > the end, or in the case where you don't want to do any evaluation
> > in
> > > > your
> > > > > > app and just want to get captures as fast as you can,
> asynchronous
> > > > could
> > > > > > increase performance there.
> > > > > >
> > > > > > I could try to implement it this way too, if you think that's a
> > worth
> > > > > > while use case, maybe just provide an 'asynchronous' flag in
> > > > > CaptureOptions?
> > > > > >
> > > > > > For doing the fileIO/compare using cordova.getThreadPool, is
> there
> > a
> > > > > > benefit to doing that if I still plan having the operation block
> > the
> > > > > > Javascript? For asynchronous mode I see why that makes sense.
> > > > > >
> > > > > > For the return values, I'll make those changes right away!
> Thanks
> > > > again
> > > > > > for providing your insight, hopefully I understood it correctly.
> > > > > >
> > > > > > -Aaron
> > > > > >
> > > > > >
> > > > > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <
> > agrieve@chromium.org
> > > > > >wrote:
> > > > > >
> > > > > >> Had a glance at your code. Things you may want to address:
> > > > > >>
> > > > > >> while(!mUIThreadDone) {}
> > > > > >>
> > > > > >> You shouldn't need to busy-wait. Pass the CallbackContext to the
> > > > > Runnable,
> > > > > >> and call success/error on it whenever you're done. No need to
> have
> > > the
> > > > > web
> > > > > >> thread wait for it. Even better would be to use a second
> Runnable
> > to
> > > > do
> > > > > >> the
> > > > > >> image compare / file IO on a background thread via
> > > > > cordova.getThreadPool()
> > > > > >>
> > > > > >> The return value to execute() should only be false if an invalid
> > > > command
> > > > > >> was passed. It's not meant to indicate the success of the
> > operation.
> > > > You
> > > > > >> should be returning true for it.
> > > > > >>
> > > > > >> Instead of return "success"/"", it might simplify things a bit
> to
> > > > > return a
> > > > > >> boolean of true/false
> > > > > >>
> > > > > >>
> > > > > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <
> > > lorin.beer.dev@gmail.com
> > > > > >> >wrote:
> > > > > >>
> > > > > >> > all the same, very impressive and faster then I expected.
> > > > > >> >
> > > > > >> > - Lorin
> > > > > >> >
> > > > > >> >
> > > > > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <
> > > > > amcharbo@gmail.com
> > > > > >> > >wrote:
> > > > > >> >
> > > > > >> > > Thanks Lorin.
> > > > > >> > >
> > > > > >> > > I did a quick test for throughput on an animating canvas. I
> > was
> > > > > able
> > > > > >> to
> > > > > >> > > grab 30 frames within a 5 seconds which equates to around
> > 166ms
> > > > per
> > > > > >> > capture
> > > > > >> > > on a Galaxy Nexus. Unfortunately not fast enough to
> automate
> > > > > >> something
> > > > > >> > > like video. As with any kind of capturing system, often the
> > > > > overhead
> > > > > >> of
> > > > > >> > > copying bits around is enough to slow down the operation of
> > the
> > > > app.
> > > > > >> > >
> > > > > >> > > -Aaron
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
> > > > > >> lorin.beer.dev@gmail.com
> > > > > >> > > >wrote:
> > > > > >> > >
> > > > > >> > > > Hey Aaron,
> > > > > >> > > >
> > > > > >> > > > very cool stuff, looking forward to checking it out!
> > > > > >> > > >
> > > > > >> > > > Question: any performance hit on the app by using Capture?
> > > > What's
> > > > > >> the
> > > > > >> > > > expected throughput on images taken in this way (given a
> > > > > particular
> > > > > >> > > device,
> > > > > >> > > > say Galaxy Nexus)?
> > > > > >> > > >
> > > > > >> > > > - Lorin
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <
> > > > > >> > amcharbo@gmail.com
> > > > > >> > > > >wrote:
> > > > > >> > > >
> > > > > >> > > > > Glad you like it :)
> > > > > >> > > > > Yes in fact the actual capture makes use of
> > > > > View.capturePicture()
> > > > > >> > which
> > > > > >> > > > > actually grabs the entire document, then that can be
> > clipped
> > > > > down
> > > > > >> to
> > > > > >> > > the
> > > > > >> > > > > size/location of a specific element.
> > > > > >> > > > >
> > > > > >> > > > > -Aaron
> > > > > >> > > > >
> > > > > >> > > > >
> > > > > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <
> > > > > >> mmocny@chromium.org>
> > > > > >> > > > > wrote:
> > > > > >> > > > >
> > > > > >> > > > > > Aaron,
> > > > > >> > > > > >
> > > > > >> > > > > > I haven't even begun looking at your implementation,
> but
> > > I'm
> > > > > >> just
> > > > > >> > > going
> > > > > >> > > > > to
> > > > > >> > > > > > say it: this is awesome!
> > > > > >> > > > > >
> > > > > >> > > > > > First question: When Capturing a DOM element, can you
> > > > capture
> > > > > >> > 'body'
> > > > > >> > > to
> > > > > >> > > > > > grab it&children for a "full content screenshot", or
> > does
> > > it
> > > > > >> have
> > > > > >> > to
> > > > > >> > > > be a
> > > > > >> > > > > > specific single element?
> > > > > >> > > > > >
> > > > > >> > > > > > -Michal
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <
> > > > > >> > > amcharbo@gmail.com
> > > > > >> > > > > > >wrote:
> > > > > >> > > > > >
> > > > > >> > > > > > > Greetings! My name is Aaron Charbonneau, happy to
> be
> > a
> > > > new
> > > > > >> > member
> > > > > >> > > of
> > > > > >> > > > > the
> > > > > >> > > > > > > mailing list!
> > > > > >> > > > > > >
> > > > > >> > > > > > > I have been developing a sceenshot plugin for
> Cordova
> > to
> > > > > help
> > > > > >> > > > > facilitate
> > > > > >> > > > > > > automation testing and debugging of Cordova apps,
> and
> > I
> > > > > would
> > > > > >> > love
> > > > > >> > > > some
> > > > > >> > > > > > > feedback on it. Currently Cordova provides a bunch
> of
> > > > > native
> > > > > >> > > > functions
> > > > > >> > > > > > > that allow you to do some cool stuff, but not much
> > > > > >> functionality
> > > > > >> > to
> > > > > >> > > > > test
> > > > > >> > > > > > > the apps that make use of them. Being able to take
> a
> > > > > capture
> > > > > >> of
> > > > > >> > > the
> > > > > >> > > > > > screen
> > > > > >> > > > > > > from within you app is a great way to automate
> testing
> > > or
> > > > > get
> > > > > >> > > > > additional
> > > > > >> > > > > > > information for debugging an issue. Since there is
> no
> > > > > >> > > > > > > Javascript mechanism for taking screen captures the
> > > > solution
> > > > > >> > would
> > > > > >> > > > have
> > > > > >> > > > > > to
> > > > > >> > > > > > > be native, which fits nicely into the "gap" that
> > > > > >> Cordova/Phonegap
> > > > > >> > > > > > bridges.
> > > > > >> > > > > > > Any medium to large scale app can benefit greatly
> > from
> > > > > >> > automation
> > > > > >> > > > > > testing
> > > > > >> > > > > > > and any app can benefit from an extra debugging
> tool,
> > > and
> > > > > >> that is
> > > > > >> > > > what
> > > > > >> > > > > I
> > > > > >> > > > > > > hope this screenshot plugin can help achieve.
> > > > > >> > > > > > >
> > > > > >> > > > > > > Currently the plugin offers 2 functions:
> > > > > >> > > > > > >
> > > > > >> > > > > > > Capture():
> > > > > >> > > > > > > * Take a capture of the current view, write that
> > capture
> > > > to
> > > > > a
> > > > > >> > .png
> > > > > >> > > > file
> > > > > >> > > > > > > with the specified file name and sub directory of
> the
> > > > sdcard
> > > > > >> > > > (fallback
> > > > > >> > > > > to
> > > > > >> > > > > > > emulated sdcard in the case there isn't an sdcard
> > > mounted)
> > > > > >> > > > > > > * Able to create a sub-screenshot with a specified
> > > > rectangle
> > > > > >> in
> > > > > >> > > order
> > > > > >> > > > > to
> > > > > >> > > > > > > block out ui elements that may be variable, and also
> > > save
> > > > > >> space.
> > > > > >> > > > > > > * Can take captures of images/dom elements
> (including
> > > > > canvas)
> > > > > >> > that
> > > > > >> > > > are
> > > > > >> > > > > > > lager than the actual screen size
> > > > > >> > > > > > >
> > > > > >> > > > > > > CaptureAndCompare():
> > > > > >> > > > > > > * All the functionality of Capture()
> > > > > >> > > > > > > * Perform a comparison between the captured image
> and
> > a
> > > > > >> baseline
> > > > > >> > > > image
> > > > > >> > > > > > > located at the specified location in either the
> assets
> > > > > folder
> > > > > >> or
> > > > > >> > > the
> > > > > >> > > > > > > sdcard.
> > > > > >> > > > > > > * User can specify per color channel tolerances as
> > well
> > > as
> > > > > >> total
> > > > > >> > > > pixel
> > > > > >> > > > > > > tolerances to avoid false positives for the
> inevitable
> > > > > >> rendering
> > > > > >> > > > > > > differences across devices.
> > > > > >> > > > > > > * Optionally output a png file that contains the
> > > > differences
> > > > > >> > > between
> > > > > >> > > > > the
> > > > > >> > > > > > > actual and the baseline for debugging/triage
> purposes,
> > > two
> > > > > >> modes:
> > > > > >> > > > > binary
> > > > > >> > > > > > > diff (all failing pixels appear as solid white) or
> the
> > > > true
> > > > > >> > > > differences
> > > > > >> > > > > > > between pixels.
> > > > > >> > > > > > >
> > > > > >> > > > > > > If you can spare some time, I would love it if you
> > could
> > > > > take
> > > > > >> a
> > > > > >> > > look
> > > > > >> > > > at
> > > > > >> > > > > > the
> > > > > >> > > > > > > api and parameters I have defined to make sure they
> > > adhere
> > > > > to
> > > > > >> > > Cordova
> > > > > >> > > > > > > plugin best practices. The most crucial part would
> be
> > > in
> > > > > the
> > > > > >> > > plugin
> > > > > >> > > > > > itself
> > > > > >> > > > > > > at ScreenCapture.java:
> > > > > >> > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > > > >> > > > > > > and captureAndCompare() are the two exposed
> functions)
> > > > > >> > > > > > >
> > > > > >> > > > > > > I'm also interested to know you thoughts on it's
> > usage.
> > > I
> > > > > >> > started
> > > > > >> > > > with
> > > > > >> > > > > > the
> > > > > >> > > > > > > mobile-spec testing framework and put some quick
> > > rendering
> > > > > >> tests
> > > > > >> > > into
> > > > > >> > > > > the
> > > > > >> > > > > > > Autotest section as a viable use case. In order to
> > get
> > > > the
> > > > > >> > WebView
> > > > > >> > > > > into
> > > > > >> > > > > > > the state I wanted to capture, I had to implement a
> > wait
> > > > > >> timer to
> > > > > >> > > > allow
> > > > > >> > > > > > the
> > > > > >> > > > > > > view to update before taking the capture. Once the
> > wait
> > > > > >> function
> > > > > >> > > > > > > completes, the capture can be taken and everything
> > from
> > > > > there
> > > > > >> is
> > > > > >> > > > > callback
> > > > > >> > > > > > > based. I use the Jasmine waitsFor()/runs() blocks
> to
> > > make
> > > > > the
> > > > > >> > > tests
> > > > > >> > > > > run
> > > > > >> > > > > > > synchronously. For some validation testing I made a
> > > > compare
> > > > > >> > > function
> > > > > >> > > > > in
> > > > > >> > > > > > > Javascript to run against the native compare. Turns
> > out
> > > > > Java
> > > > > >> > runs
> > > > > >> > > > alot
> > > > > >> > > > > > > faster than Javascript (surprise!) thus the native
> > > compare
> > > > > is
> > > > > >> > about
> > > > > >> > > > > 2-5x
> > > > > >> > > > > > > faster. All in all the process is fairly quick, the
> > > full
> > > > > >> > > > > > captureAndCompare
> > > > > >> > > > > > > with 3 file io's takes about 233ms on a Nexus One,
> and
> > > > > ~100ms
> > > > > >> on
> > > > > >> > a
> > > > > >> > > > > Nexus
> > > > > >> > > > > > 10
> > > > > >> > > > > > > (256x256 image size).
> > > > > >> > > > > > >
> > > > > >> > > > > > > Anyways here is the usage:
> > > > > >> > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > > > > >> > > > > > >
> > > > > >> > > > > > > And here's the JS wrapper for the plugin calls:
> > > > > >> > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > > > > >> > > > > > >
> > > > > >> > > > > > > Thanks for your time, and I look forward to ANY
> > feedback
> > > > > >> positive
> > > > > >> > > or
> > > > > >> > > > > > > negative.
> > > > > >> > > > > > >
> > > > > >> > > > > > > Thanks,
> > > > > >> > > > > > > Aaron
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Aaron Charbonneau <am...@gmail.com>.
Yah that's how I have it working now :) No problem with that approach for
working with Jasmine.
I made reference to running them asynchronously to respond to Andrew about
returning immediately after the bits are captured, which in the case of
using that with Jasmine would only see a benefit IF we could run multiple
'it' blocks at the same time.
I think however Andrew was thinking of a different use case where we aren't
limited by a framework that requires tests to run serially, and results can
just come back as they are ready, and be evaluated all together at the end,
or not at all if you just want the captures and no evaluation.
-Aaron
On Wed, Mar 13, 2013 at 1:53 PM, Braden Shepherdson <br...@chromium.org>wrote:
> As far as I know, you're correct: there's no way to move on the next it()
> block before the first one is done.
>
> What's the problem with just making the tests wait? Why does the next it()
> block need to start early? You can make your call to Java, use waitFor to
> wait until the callback with the data is called, run your expectations, and
> then move on to the next it() block. If the next it() block is using the
> result of this one, then they should be one it() block.
>
> Braden
>
>
> On Wed, Mar 13, 2013 at 4:46 PM, Aaron Charbonneau <amcharbo@gmail.com
> >wrote:
>
> > Hi Braden,
> > Sorry maybe I am overlooking something. The way I understand it is that
> a
> > previous 'it' block needs to complete before the next one can run. So I
> > have various tests setup like this:
> >
> > it("renders something", function() {
> > //draw something
> > ....
> > captureAndCompare(); //call the native function
> > waitsFor(function() {
> > return captureComplete; //set by the native function callback
> > }, "capture never completed", 10000);
> > runs(function() {
> > expect(captureResult).toBe(0); //expects difference to be 0
> > });
> > });
> > //next 'it' test
> > ...
> >
> > The expect block needs to wait until there is a result returned from the
> > native captureAndCompare call. Therefore we use a waitsFor/runs block
> > there.
> >
> > But say we wanted to allow Javascript to run immediately after calling
> that
> > native function. The only thing you need to wait for is that Java
> grabbed
> > the bits, and the rest of the processing will be done 'at some point in
> the
> > future.' So as Andrew suggested you would return as soon as the bits are
> > captured, and allow JS to run again. The problem is that our expect
> block
> > needs the results, so that has to wait, the only other thing to do would
> be
> > to run the next 'it' block.
> >
> > This is the part I didn't think was possible, can the next 'it' block be
> > run while the previous 'it' is still waiting to execute the 'expect'
> block?
> > Or is there another approach all together that would allow this?
> >
> > Thanks,
> > Aaron
> >
> >
> > On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <braden@chromium.org
> > >wrote:
> >
> > > Jasmine does support asynchronous tests, and waiting an arbitrary
> period
> > or
> > > until some condition is true. Why does that not work for these tests?
> > >
> > > Braden
> > >
> > >
> > > On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <amcharbo@gmail.com
> > > >wrote:
> > >
> > > > Quick follow up question for the return values stuff:
> > > > Does it make sense to have my functions called from execute
> (capture()
> > > and
> > > > captureAndCompare()) return an error string if one is encountered and
> > > pass
> > > > that back in callbackContext.error, then have execute return true?
> > > >
> > > > Or is there a preferred way to get errors from sub functions back
> into
> > > > Javascript?
> > > >
> > > > Thanks!
> > > >
> > > >
> > > > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <
> > amcharbo@gmail.com
> > > > >wrote:
> > > >
> > > > > Thanks Andrew these are great suggestions!
> > > > >
> > > > > About not needing the busy-wait in getScreenBits:
> > > > > I'm not a fan of busy-waiting either.
> > > > > Perhaps it's a side effect of Jasmine requiring tests to be run and
> > > > > evaluated synchronously that I decided to use a busy-wait in
> > > > getScreenBits.
> > > > > If the Runnable calls CallbackContext.success/error once the bits
> > are
> > > > > captured (but not written to file) on the UIThread, the Javascript
> > side
> > > > > will proceed to run before it has an image url/compare result, most
> > > > likely
> > > > > causing that test to fail. As far as I know Jasmine doesn't allow
> > for
> > > > > kicking off a bunch of tests and then gather all the results at the
> > end
> > > > and
> > > > > output pass/fails (each 'expect' function needs to be within an
> 'it'
> > > > block,
> > > > > essentially that means one test must be evaluated before the next
> can
> > > > > begin).
> > > > >
> > > > > I can see the usefulness of allowing the capture, file io, and
> > > comparison
> > > > > to be able to run asynchronously from the Javascript though. In the
> > > case
> > > > > where you have your own test framework that CAN kick off a bunch of
> > > tests
> > > > > and just gather the results as they come in and do a batch
> evaluation
> > > at
> > > > > the end, or in the case where you don't want to do any evaluation
> in
> > > your
> > > > > app and just want to get captures as fast as you can, asynchronous
> > > could
> > > > > increase performance there.
> > > > >
> > > > > I could try to implement it this way too, if you think that's a
> worth
> > > > > while use case, maybe just provide an 'asynchronous' flag in
> > > > CaptureOptions?
> > > > >
> > > > > For doing the fileIO/compare using cordova.getThreadPool, is there
> a
> > > > > benefit to doing that if I still plan having the operation block
> the
> > > > > Javascript? For asynchronous mode I see why that makes sense.
> > > > >
> > > > > For the return values, I'll make those changes right away! Thanks
> > > again
> > > > > for providing your insight, hopefully I understood it correctly.
> > > > >
> > > > > -Aaron
> > > > >
> > > > >
> > > > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <
> agrieve@chromium.org
> > > > >wrote:
> > > > >
> > > > >> Had a glance at your code. Things you may want to address:
> > > > >>
> > > > >> while(!mUIThreadDone) {}
> > > > >>
> > > > >> You shouldn't need to busy-wait. Pass the CallbackContext to the
> > > > Runnable,
> > > > >> and call success/error on it whenever you're done. No need to have
> > the
> > > > web
> > > > >> thread wait for it. Even better would be to use a second Runnable
> to
> > > do
> > > > >> the
> > > > >> image compare / file IO on a background thread via
> > > > cordova.getThreadPool()
> > > > >>
> > > > >> The return value to execute() should only be false if an invalid
> > > command
> > > > >> was passed. It's not meant to indicate the success of the
> operation.
> > > You
> > > > >> should be returning true for it.
> > > > >>
> > > > >> Instead of return "success"/"", it might simplify things a bit to
> > > > return a
> > > > >> boolean of true/false
> > > > >>
> > > > >>
> > > > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <
> > lorin.beer.dev@gmail.com
> > > > >> >wrote:
> > > > >>
> > > > >> > all the same, very impressive and faster then I expected.
> > > > >> >
> > > > >> > - Lorin
> > > > >> >
> > > > >> >
> > > > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <
> > > > amcharbo@gmail.com
> > > > >> > >wrote:
> > > > >> >
> > > > >> > > Thanks Lorin.
> > > > >> > >
> > > > >> > > I did a quick test for throughput on an animating canvas. I
> was
> > > > able
> > > > >> to
> > > > >> > > grab 30 frames within a 5 seconds which equates to around
> 166ms
> > > per
> > > > >> > capture
> > > > >> > > on a Galaxy Nexus. Unfortunately not fast enough to automate
> > > > >> something
> > > > >> > > like video. As with any kind of capturing system, often the
> > > > overhead
> > > > >> of
> > > > >> > > copying bits around is enough to slow down the operation of
> the
> > > app.
> > > > >> > >
> > > > >> > > -Aaron
> > > > >> > >
> > > > >> > >
> > > > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
> > > > >> lorin.beer.dev@gmail.com
> > > > >> > > >wrote:
> > > > >> > >
> > > > >> > > > Hey Aaron,
> > > > >> > > >
> > > > >> > > > very cool stuff, looking forward to checking it out!
> > > > >> > > >
> > > > >> > > > Question: any performance hit on the app by using Capture?
> > > What's
> > > > >> the
> > > > >> > > > expected throughput on images taken in this way (given a
> > > > particular
> > > > >> > > device,
> > > > >> > > > say Galaxy Nexus)?
> > > > >> > > >
> > > > >> > > > - Lorin
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <
> > > > >> > amcharbo@gmail.com
> > > > >> > > > >wrote:
> > > > >> > > >
> > > > >> > > > > Glad you like it :)
> > > > >> > > > > Yes in fact the actual capture makes use of
> > > > View.capturePicture()
> > > > >> > which
> > > > >> > > > > actually grabs the entire document, then that can be
> clipped
> > > > down
> > > > >> to
> > > > >> > > the
> > > > >> > > > > size/location of a specific element.
> > > > >> > > > >
> > > > >> > > > > -Aaron
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <
> > > > >> mmocny@chromium.org>
> > > > >> > > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Aaron,
> > > > >> > > > > >
> > > > >> > > > > > I haven't even begun looking at your implementation, but
> > I'm
> > > > >> just
> > > > >> > > going
> > > > >> > > > > to
> > > > >> > > > > > say it: this is awesome!
> > > > >> > > > > >
> > > > >> > > > > > First question: When Capturing a DOM element, can you
> > > capture
> > > > >> > 'body'
> > > > >> > > to
> > > > >> > > > > > grab it&children for a "full content screenshot", or
> does
> > it
> > > > >> have
> > > > >> > to
> > > > >> > > > be a
> > > > >> > > > > > specific single element?
> > > > >> > > > > >
> > > > >> > > > > > -Michal
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <
> > > > >> > > amcharbo@gmail.com
> > > > >> > > > > > >wrote:
> > > > >> > > > > >
> > > > >> > > > > > > Greetings! My name is Aaron Charbonneau, happy to be
> a
> > > new
> > > > >> > member
> > > > >> > > of
> > > > >> > > > > the
> > > > >> > > > > > > mailing list!
> > > > >> > > > > > >
> > > > >> > > > > > > I have been developing a sceenshot plugin for Cordova
> to
> > > > help
> > > > >> > > > > facilitate
> > > > >> > > > > > > automation testing and debugging of Cordova apps, and
> I
> > > > would
> > > > >> > love
> > > > >> > > > some
> > > > >> > > > > > > feedback on it. Currently Cordova provides a bunch of
> > > > native
> > > > >> > > > functions
> > > > >> > > > > > > that allow you to do some cool stuff, but not much
> > > > >> functionality
> > > > >> > to
> > > > >> > > > > test
> > > > >> > > > > > > the apps that make use of them. Being able to take a
> > > > capture
> > > > >> of
> > > > >> > > the
> > > > >> > > > > > screen
> > > > >> > > > > > > from within you app is a great way to automate testing
> > or
> > > > get
> > > > >> > > > > additional
> > > > >> > > > > > > information for debugging an issue. Since there is no
> > > > >> > > > > > > Javascript mechanism for taking screen captures the
> > > solution
> > > > >> > would
> > > > >> > > > have
> > > > >> > > > > > to
> > > > >> > > > > > > be native, which fits nicely into the "gap" that
> > > > >> Cordova/Phonegap
> > > > >> > > > > > bridges.
> > > > >> > > > > > > Any medium to large scale app can benefit greatly
> from
> > > > >> > automation
> > > > >> > > > > > testing
> > > > >> > > > > > > and any app can benefit from an extra debugging tool,
> > and
> > > > >> that is
> > > > >> > > > what
> > > > >> > > > > I
> > > > >> > > > > > > hope this screenshot plugin can help achieve.
> > > > >> > > > > > >
> > > > >> > > > > > > Currently the plugin offers 2 functions:
> > > > >> > > > > > >
> > > > >> > > > > > > Capture():
> > > > >> > > > > > > * Take a capture of the current view, write that
> capture
> > > to
> > > > a
> > > > >> > .png
> > > > >> > > > file
> > > > >> > > > > > > with the specified file name and sub directory of the
> > > sdcard
> > > > >> > > > (fallback
> > > > >> > > > > to
> > > > >> > > > > > > emulated sdcard in the case there isn't an sdcard
> > mounted)
> > > > >> > > > > > > * Able to create a sub-screenshot with a specified
> > > rectangle
> > > > >> in
> > > > >> > > order
> > > > >> > > > > to
> > > > >> > > > > > > block out ui elements that may be variable, and also
> > save
> > > > >> space.
> > > > >> > > > > > > * Can take captures of images/dom elements (including
> > > > canvas)
> > > > >> > that
> > > > >> > > > are
> > > > >> > > > > > > lager than the actual screen size
> > > > >> > > > > > >
> > > > >> > > > > > > CaptureAndCompare():
> > > > >> > > > > > > * All the functionality of Capture()
> > > > >> > > > > > > * Perform a comparison between the captured image and
> a
> > > > >> baseline
> > > > >> > > > image
> > > > >> > > > > > > located at the specified location in either the assets
> > > > folder
> > > > >> or
> > > > >> > > the
> > > > >> > > > > > > sdcard.
> > > > >> > > > > > > * User can specify per color channel tolerances as
> well
> > as
> > > > >> total
> > > > >> > > > pixel
> > > > >> > > > > > > tolerances to avoid false positives for the inevitable
> > > > >> rendering
> > > > >> > > > > > > differences across devices.
> > > > >> > > > > > > * Optionally output a png file that contains the
> > > differences
> > > > >> > > between
> > > > >> > > > > the
> > > > >> > > > > > > actual and the baseline for debugging/triage purposes,
> > two
> > > > >> modes:
> > > > >> > > > > binary
> > > > >> > > > > > > diff (all failing pixels appear as solid white) or the
> > > true
> > > > >> > > > differences
> > > > >> > > > > > > between pixels.
> > > > >> > > > > > >
> > > > >> > > > > > > If you can spare some time, I would love it if you
> could
> > > > take
> > > > >> a
> > > > >> > > look
> > > > >> > > > at
> > > > >> > > > > > the
> > > > >> > > > > > > api and parameters I have defined to make sure they
> > adhere
> > > > to
> > > > >> > > Cordova
> > > > >> > > > > > > plugin best practices. The most crucial part would be
> > in
> > > > the
> > > > >> > > plugin
> > > > >> > > > > > itself
> > > > >> > > > > > > at ScreenCapture.java:
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > > >> > > > > > > and captureAndCompare() are the two exposed functions)
> > > > >> > > > > > >
> > > > >> > > > > > > I'm also interested to know you thoughts on it's
> usage.
> > I
> > > > >> > started
> > > > >> > > > with
> > > > >> > > > > > the
> > > > >> > > > > > > mobile-spec testing framework and put some quick
> > rendering
> > > > >> tests
> > > > >> > > into
> > > > >> > > > > the
> > > > >> > > > > > > Autotest section as a viable use case. In order to
> get
> > > the
> > > > >> > WebView
> > > > >> > > > > into
> > > > >> > > > > > > the state I wanted to capture, I had to implement a
> wait
> > > > >> timer to
> > > > >> > > > allow
> > > > >> > > > > > the
> > > > >> > > > > > > view to update before taking the capture. Once the
> wait
> > > > >> function
> > > > >> > > > > > > completes, the capture can be taken and everything
> from
> > > > there
> > > > >> is
> > > > >> > > > > callback
> > > > >> > > > > > > based. I use the Jasmine waitsFor()/runs() blocks to
> > make
> > > > the
> > > > >> > > tests
> > > > >> > > > > run
> > > > >> > > > > > > synchronously. For some validation testing I made a
> > > compare
> > > > >> > > function
> > > > >> > > > > in
> > > > >> > > > > > > Javascript to run against the native compare. Turns
> out
> > > > Java
> > > > >> > runs
> > > > >> > > > alot
> > > > >> > > > > > > faster than Javascript (surprise!) thus the native
> > compare
> > > > is
> > > > >> > about
> > > > >> > > > > 2-5x
> > > > >> > > > > > > faster. All in all the process is fairly quick, the
> > full
> > > > >> > > > > > captureAndCompare
> > > > >> > > > > > > with 3 file io's takes about 233ms on a Nexus One, and
> > > > ~100ms
> > > > >> on
> > > > >> > a
> > > > >> > > > > Nexus
> > > > >> > > > > > 10
> > > > >> > > > > > > (256x256 image size).
> > > > >> > > > > > >
> > > > >> > > > > > > Anyways here is the usage:
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > > > >> > > > > > >
> > > > >> > > > > > > And here's the JS wrapper for the plugin calls:
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > > > >> > > > > > >
> > > > >> > > > > > > Thanks for your time, and I look forward to ANY
> feedback
> > > > >> positive
> > > > >> > > or
> > > > >> > > > > > > negative.
> > > > >> > > > > > >
> > > > >> > > > > > > Thanks,
> > > > >> > > > > > > Aaron
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Braden Shepherdson <br...@chromium.org>.
As far as I know, you're correct: there's no way to move on the next it()
block before the first one is done.
What's the problem with just making the tests wait? Why does the next it()
block need to start early? You can make your call to Java, use waitFor to
wait until the callback with the data is called, run your expectations, and
then move on to the next it() block. If the next it() block is using the
result of this one, then they should be one it() block.
Braden
On Wed, Mar 13, 2013 at 4:46 PM, Aaron Charbonneau <am...@gmail.com>wrote:
> Hi Braden,
> Sorry maybe I am overlooking something. The way I understand it is that a
> previous 'it' block needs to complete before the next one can run. So I
> have various tests setup like this:
>
> it("renders something", function() {
> //draw something
> ....
> captureAndCompare(); //call the native function
> waitsFor(function() {
> return captureComplete; //set by the native function callback
> }, "capture never completed", 10000);
> runs(function() {
> expect(captureResult).toBe(0); //expects difference to be 0
> });
> });
> //next 'it' test
> ...
>
> The expect block needs to wait until there is a result returned from the
> native captureAndCompare call. Therefore we use a waitsFor/runs block
> there.
>
> But say we wanted to allow Javascript to run immediately after calling that
> native function. The only thing you need to wait for is that Java grabbed
> the bits, and the rest of the processing will be done 'at some point in the
> future.' So as Andrew suggested you would return as soon as the bits are
> captured, and allow JS to run again. The problem is that our expect block
> needs the results, so that has to wait, the only other thing to do would be
> to run the next 'it' block.
>
> This is the part I didn't think was possible, can the next 'it' block be
> run while the previous 'it' is still waiting to execute the 'expect' block?
> Or is there another approach all together that would allow this?
>
> Thanks,
> Aaron
>
>
> On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <braden@chromium.org
> >wrote:
>
> > Jasmine does support asynchronous tests, and waiting an arbitrary period
> or
> > until some condition is true. Why does that not work for these tests?
> >
> > Braden
> >
> >
> > On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <amcharbo@gmail.com
> > >wrote:
> >
> > > Quick follow up question for the return values stuff:
> > > Does it make sense to have my functions called from execute (capture()
> > and
> > > captureAndCompare()) return an error string if one is encountered and
> > pass
> > > that back in callbackContext.error, then have execute return true?
> > >
> > > Or is there a preferred way to get errors from sub functions back into
> > > Javascript?
> > >
> > > Thanks!
> > >
> > >
> > > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <
> amcharbo@gmail.com
> > > >wrote:
> > >
> > > > Thanks Andrew these are great suggestions!
> > > >
> > > > About not needing the busy-wait in getScreenBits:
> > > > I'm not a fan of busy-waiting either.
> > > > Perhaps it's a side effect of Jasmine requiring tests to be run and
> > > > evaluated synchronously that I decided to use a busy-wait in
> > > getScreenBits.
> > > > If the Runnable calls CallbackContext.success/error once the bits
> are
> > > > captured (but not written to file) on the UIThread, the Javascript
> side
> > > > will proceed to run before it has an image url/compare result, most
> > > likely
> > > > causing that test to fail. As far as I know Jasmine doesn't allow
> for
> > > > kicking off a bunch of tests and then gather all the results at the
> end
> > > and
> > > > output pass/fails (each 'expect' function needs to be within an 'it'
> > > block,
> > > > essentially that means one test must be evaluated before the next can
> > > > begin).
> > > >
> > > > I can see the usefulness of allowing the capture, file io, and
> > comparison
> > > > to be able to run asynchronously from the Javascript though. In the
> > case
> > > > where you have your own test framework that CAN kick off a bunch of
> > tests
> > > > and just gather the results as they come in and do a batch evaluation
> > at
> > > > the end, or in the case where you don't want to do any evaluation in
> > your
> > > > app and just want to get captures as fast as you can, asynchronous
> > could
> > > > increase performance there.
> > > >
> > > > I could try to implement it this way too, if you think that's a worth
> > > > while use case, maybe just provide an 'asynchronous' flag in
> > > CaptureOptions?
> > > >
> > > > For doing the fileIO/compare using cordova.getThreadPool, is there a
> > > > benefit to doing that if I still plan having the operation block the
> > > > Javascript? For asynchronous mode I see why that makes sense.
> > > >
> > > > For the return values, I'll make those changes right away! Thanks
> > again
> > > > for providing your insight, hopefully I understood it correctly.
> > > >
> > > > -Aaron
> > > >
> > > >
> > > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <agrieve@chromium.org
> > > >wrote:
> > > >
> > > >> Had a glance at your code. Things you may want to address:
> > > >>
> > > >> while(!mUIThreadDone) {}
> > > >>
> > > >> You shouldn't need to busy-wait. Pass the CallbackContext to the
> > > Runnable,
> > > >> and call success/error on it whenever you're done. No need to have
> the
> > > web
> > > >> thread wait for it. Even better would be to use a second Runnable to
> > do
> > > >> the
> > > >> image compare / file IO on a background thread via
> > > cordova.getThreadPool()
> > > >>
> > > >> The return value to execute() should only be false if an invalid
> > command
> > > >> was passed. It's not meant to indicate the success of the operation.
> > You
> > > >> should be returning true for it.
> > > >>
> > > >> Instead of return "success"/"", it might simplify things a bit to
> > > return a
> > > >> boolean of true/false
> > > >>
> > > >>
> > > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <
> lorin.beer.dev@gmail.com
> > > >> >wrote:
> > > >>
> > > >> > all the same, very impressive and faster then I expected.
> > > >> >
> > > >> > - Lorin
> > > >> >
> > > >> >
> > > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <
> > > amcharbo@gmail.com
> > > >> > >wrote:
> > > >> >
> > > >> > > Thanks Lorin.
> > > >> > >
> > > >> > > I did a quick test for throughput on an animating canvas. I was
> > > able
> > > >> to
> > > >> > > grab 30 frames within a 5 seconds which equates to around 166ms
> > per
> > > >> > capture
> > > >> > > on a Galaxy Nexus. Unfortunately not fast enough to automate
> > > >> something
> > > >> > > like video. As with any kind of capturing system, often the
> > > overhead
> > > >> of
> > > >> > > copying bits around is enough to slow down the operation of the
> > app.
> > > >> > >
> > > >> > > -Aaron
> > > >> > >
> > > >> > >
> > > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
> > > >> lorin.beer.dev@gmail.com
> > > >> > > >wrote:
> > > >> > >
> > > >> > > > Hey Aaron,
> > > >> > > >
> > > >> > > > very cool stuff, looking forward to checking it out!
> > > >> > > >
> > > >> > > > Question: any performance hit on the app by using Capture?
> > What's
> > > >> the
> > > >> > > > expected throughput on images taken in this way (given a
> > > particular
> > > >> > > device,
> > > >> > > > say Galaxy Nexus)?
> > > >> > > >
> > > >> > > > - Lorin
> > > >> > > >
> > > >> > > >
> > > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <
> > > >> > amcharbo@gmail.com
> > > >> > > > >wrote:
> > > >> > > >
> > > >> > > > > Glad you like it :)
> > > >> > > > > Yes in fact the actual capture makes use of
> > > View.capturePicture()
> > > >> > which
> > > >> > > > > actually grabs the entire document, then that can be clipped
> > > down
> > > >> to
> > > >> > > the
> > > >> > > > > size/location of a specific element.
> > > >> > > > >
> > > >> > > > > -Aaron
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <
> > > >> mmocny@chromium.org>
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Aaron,
> > > >> > > > > >
> > > >> > > > > > I haven't even begun looking at your implementation, but
> I'm
> > > >> just
> > > >> > > going
> > > >> > > > > to
> > > >> > > > > > say it: this is awesome!
> > > >> > > > > >
> > > >> > > > > > First question: When Capturing a DOM element, can you
> > capture
> > > >> > 'body'
> > > >> > > to
> > > >> > > > > > grab it&children for a "full content screenshot", or does
> it
> > > >> have
> > > >> > to
> > > >> > > > be a
> > > >> > > > > > specific single element?
> > > >> > > > > >
> > > >> > > > > > -Michal
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <
> > > >> > > amcharbo@gmail.com
> > > >> > > > > > >wrote:
> > > >> > > > > >
> > > >> > > > > > > Greetings! My name is Aaron Charbonneau, happy to be a
> > new
> > > >> > member
> > > >> > > of
> > > >> > > > > the
> > > >> > > > > > > mailing list!
> > > >> > > > > > >
> > > >> > > > > > > I have been developing a sceenshot plugin for Cordova to
> > > help
> > > >> > > > > facilitate
> > > >> > > > > > > automation testing and debugging of Cordova apps, and I
> > > would
> > > >> > love
> > > >> > > > some
> > > >> > > > > > > feedback on it. Currently Cordova provides a bunch of
> > > native
> > > >> > > > functions
> > > >> > > > > > > that allow you to do some cool stuff, but not much
> > > >> functionality
> > > >> > to
> > > >> > > > > test
> > > >> > > > > > > the apps that make use of them. Being able to take a
> > > capture
> > > >> of
> > > >> > > the
> > > >> > > > > > screen
> > > >> > > > > > > from within you app is a great way to automate testing
> or
> > > get
> > > >> > > > > additional
> > > >> > > > > > > information for debugging an issue. Since there is no
> > > >> > > > > > > Javascript mechanism for taking screen captures the
> > solution
> > > >> > would
> > > >> > > > have
> > > >> > > > > > to
> > > >> > > > > > > be native, which fits nicely into the "gap" that
> > > >> Cordova/Phonegap
> > > >> > > > > > bridges.
> > > >> > > > > > > Any medium to large scale app can benefit greatly from
> > > >> > automation
> > > >> > > > > > testing
> > > >> > > > > > > and any app can benefit from an extra debugging tool,
> and
> > > >> that is
> > > >> > > > what
> > > >> > > > > I
> > > >> > > > > > > hope this screenshot plugin can help achieve.
> > > >> > > > > > >
> > > >> > > > > > > Currently the plugin offers 2 functions:
> > > >> > > > > > >
> > > >> > > > > > > Capture():
> > > >> > > > > > > * Take a capture of the current view, write that capture
> > to
> > > a
> > > >> > .png
> > > >> > > > file
> > > >> > > > > > > with the specified file name and sub directory of the
> > sdcard
> > > >> > > > (fallback
> > > >> > > > > to
> > > >> > > > > > > emulated sdcard in the case there isn't an sdcard
> mounted)
> > > >> > > > > > > * Able to create a sub-screenshot with a specified
> > rectangle
> > > >> in
> > > >> > > order
> > > >> > > > > to
> > > >> > > > > > > block out ui elements that may be variable, and also
> save
> > > >> space.
> > > >> > > > > > > * Can take captures of images/dom elements (including
> > > canvas)
> > > >> > that
> > > >> > > > are
> > > >> > > > > > > lager than the actual screen size
> > > >> > > > > > >
> > > >> > > > > > > CaptureAndCompare():
> > > >> > > > > > > * All the functionality of Capture()
> > > >> > > > > > > * Perform a comparison between the captured image and a
> > > >> baseline
> > > >> > > > image
> > > >> > > > > > > located at the specified location in either the assets
> > > folder
> > > >> or
> > > >> > > the
> > > >> > > > > > > sdcard.
> > > >> > > > > > > * User can specify per color channel tolerances as well
> as
> > > >> total
> > > >> > > > pixel
> > > >> > > > > > > tolerances to avoid false positives for the inevitable
> > > >> rendering
> > > >> > > > > > > differences across devices.
> > > >> > > > > > > * Optionally output a png file that contains the
> > differences
> > > >> > > between
> > > >> > > > > the
> > > >> > > > > > > actual and the baseline for debugging/triage purposes,
> two
> > > >> modes:
> > > >> > > > > binary
> > > >> > > > > > > diff (all failing pixels appear as solid white) or the
> > true
> > > >> > > > differences
> > > >> > > > > > > between pixels.
> > > >> > > > > > >
> > > >> > > > > > > If you can spare some time, I would love it if you could
> > > take
> > > >> a
> > > >> > > look
> > > >> > > > at
> > > >> > > > > > the
> > > >> > > > > > > api and parameters I have defined to make sure they
> adhere
> > > to
> > > >> > > Cordova
> > > >> > > > > > > plugin best practices. The most crucial part would be
> in
> > > the
> > > >> > > plugin
> > > >> > > > > > itself
> > > >> > > > > > > at ScreenCapture.java:
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > >> > > > > > > and captureAndCompare() are the two exposed functions)
> > > >> > > > > > >
> > > >> > > > > > > I'm also interested to know you thoughts on it's usage.
> I
> > > >> > started
> > > >> > > > with
> > > >> > > > > > the
> > > >> > > > > > > mobile-spec testing framework and put some quick
> rendering
> > > >> tests
> > > >> > > into
> > > >> > > > > the
> > > >> > > > > > > Autotest section as a viable use case. In order to get
> > the
> > > >> > WebView
> > > >> > > > > into
> > > >> > > > > > > the state I wanted to capture, I had to implement a wait
> > > >> timer to
> > > >> > > > allow
> > > >> > > > > > the
> > > >> > > > > > > view to update before taking the capture. Once the wait
> > > >> function
> > > >> > > > > > > completes, the capture can be taken and everything from
> > > there
> > > >> is
> > > >> > > > > callback
> > > >> > > > > > > based. I use the Jasmine waitsFor()/runs() blocks to
> make
> > > the
> > > >> > > tests
> > > >> > > > > run
> > > >> > > > > > > synchronously. For some validation testing I made a
> > compare
> > > >> > > function
> > > >> > > > > in
> > > >> > > > > > > Javascript to run against the native compare. Turns out
> > > Java
> > > >> > runs
> > > >> > > > alot
> > > >> > > > > > > faster than Javascript (surprise!) thus the native
> compare
> > > is
> > > >> > about
> > > >> > > > > 2-5x
> > > >> > > > > > > faster. All in all the process is fairly quick, the
> full
> > > >> > > > > > captureAndCompare
> > > >> > > > > > > with 3 file io's takes about 233ms on a Nexus One, and
> > > ~100ms
> > > >> on
> > > >> > a
> > > >> > > > > Nexus
> > > >> > > > > > 10
> > > >> > > > > > > (256x256 image size).
> > > >> > > > > > >
> > > >> > > > > > > Anyways here is the usage:
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > > >> > > > > > >
> > > >> > > > > > > And here's the JS wrapper for the plugin calls:
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > > >> > > > > > >
> > > >> > > > > > > Thanks for your time, and I look forward to ANY feedback
> > > >> positive
> > > >> > > or
> > > >> > > > > > > negative.
> > > >> > > > > > >
> > > >> > > > > > > Thanks,
> > > >> > > > > > > Aaron
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Aaron Charbonneau <am...@gmail.com>.
Hi Braden,
Sorry maybe I am overlooking something. The way I understand it is that a
previous 'it' block needs to complete before the next one can run. So I
have various tests setup like this:
it("renders something", function() {
//draw something
....
captureAndCompare(); //call the native function
waitsFor(function() {
return captureComplete; //set by the native function callback
}, "capture never completed", 10000);
runs(function() {
expect(captureResult).toBe(0); //expects difference to be 0
});
});
//next 'it' test
...
The expect block needs to wait until there is a result returned from the
native captureAndCompare call. Therefore we use a waitsFor/runs block
there.
But say we wanted to allow Javascript to run immediately after calling that
native function. The only thing you need to wait for is that Java grabbed
the bits, and the rest of the processing will be done 'at some point in the
future.' So as Andrew suggested you would return as soon as the bits are
captured, and allow JS to run again. The problem is that our expect block
needs the results, so that has to wait, the only other thing to do would be
to run the next 'it' block.
This is the part I didn't think was possible, can the next 'it' block be
run while the previous 'it' is still waiting to execute the 'expect' block?
Or is there another approach all together that would allow this?
Thanks,
Aaron
On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <br...@chromium.org>wrote:
> Jasmine does support asynchronous tests, and waiting an arbitrary period or
> until some condition is true. Why does that not work for these tests?
>
> Braden
>
>
> On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <amcharbo@gmail.com
> >wrote:
>
> > Quick follow up question for the return values stuff:
> > Does it make sense to have my functions called from execute (capture()
> and
> > captureAndCompare()) return an error string if one is encountered and
> pass
> > that back in callbackContext.error, then have execute return true?
> >
> > Or is there a preferred way to get errors from sub functions back into
> > Javascript?
> >
> > Thanks!
> >
> >
> > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <amcharbo@gmail.com
> > >wrote:
> >
> > > Thanks Andrew these are great suggestions!
> > >
> > > About not needing the busy-wait in getScreenBits:
> > > I'm not a fan of busy-waiting either.
> > > Perhaps it's a side effect of Jasmine requiring tests to be run and
> > > evaluated synchronously that I decided to use a busy-wait in
> > getScreenBits.
> > > If the Runnable calls CallbackContext.success/error once the bits are
> > > captured (but not written to file) on the UIThread, the Javascript side
> > > will proceed to run before it has an image url/compare result, most
> > likely
> > > causing that test to fail. As far as I know Jasmine doesn't allow for
> > > kicking off a bunch of tests and then gather all the results at the end
> > and
> > > output pass/fails (each 'expect' function needs to be within an 'it'
> > block,
> > > essentially that means one test must be evaluated before the next can
> > > begin).
> > >
> > > I can see the usefulness of allowing the capture, file io, and
> comparison
> > > to be able to run asynchronously from the Javascript though. In the
> case
> > > where you have your own test framework that CAN kick off a bunch of
> tests
> > > and just gather the results as they come in and do a batch evaluation
> at
> > > the end, or in the case where you don't want to do any evaluation in
> your
> > > app and just want to get captures as fast as you can, asynchronous
> could
> > > increase performance there.
> > >
> > > I could try to implement it this way too, if you think that's a worth
> > > while use case, maybe just provide an 'asynchronous' flag in
> > CaptureOptions?
> > >
> > > For doing the fileIO/compare using cordova.getThreadPool, is there a
> > > benefit to doing that if I still plan having the operation block the
> > > Javascript? For asynchronous mode I see why that makes sense.
> > >
> > > For the return values, I'll make those changes right away! Thanks
> again
> > > for providing your insight, hopefully I understood it correctly.
> > >
> > > -Aaron
> > >
> > >
> > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <agrieve@chromium.org
> > >wrote:
> > >
> > >> Had a glance at your code. Things you may want to address:
> > >>
> > >> while(!mUIThreadDone) {}
> > >>
> > >> You shouldn't need to busy-wait. Pass the CallbackContext to the
> > Runnable,
> > >> and call success/error on it whenever you're done. No need to have the
> > web
> > >> thread wait for it. Even better would be to use a second Runnable to
> do
> > >> the
> > >> image compare / file IO on a background thread via
> > cordova.getThreadPool()
> > >>
> > >> The return value to execute() should only be false if an invalid
> command
> > >> was passed. It's not meant to indicate the success of the operation.
> You
> > >> should be returning true for it.
> > >>
> > >> Instead of return "success"/"", it might simplify things a bit to
> > return a
> > >> boolean of true/false
> > >>
> > >>
> > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <lorin.beer.dev@gmail.com
> > >> >wrote:
> > >>
> > >> > all the same, very impressive and faster then I expected.
> > >> >
> > >> > - Lorin
> > >> >
> > >> >
> > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <
> > amcharbo@gmail.com
> > >> > >wrote:
> > >> >
> > >> > > Thanks Lorin.
> > >> > >
> > >> > > I did a quick test for throughput on an animating canvas. I was
> > able
> > >> to
> > >> > > grab 30 frames within a 5 seconds which equates to around 166ms
> per
> > >> > capture
> > >> > > on a Galaxy Nexus. Unfortunately not fast enough to automate
> > >> something
> > >> > > like video. As with any kind of capturing system, often the
> > overhead
> > >> of
> > >> > > copying bits around is enough to slow down the operation of the
> app.
> > >> > >
> > >> > > -Aaron
> > >> > >
> > >> > >
> > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
> > >> lorin.beer.dev@gmail.com
> > >> > > >wrote:
> > >> > >
> > >> > > > Hey Aaron,
> > >> > > >
> > >> > > > very cool stuff, looking forward to checking it out!
> > >> > > >
> > >> > > > Question: any performance hit on the app by using Capture?
> What's
> > >> the
> > >> > > > expected throughput on images taken in this way (given a
> > particular
> > >> > > device,
> > >> > > > say Galaxy Nexus)?
> > >> > > >
> > >> > > > - Lorin
> > >> > > >
> > >> > > >
> > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <
> > >> > amcharbo@gmail.com
> > >> > > > >wrote:
> > >> > > >
> > >> > > > > Glad you like it :)
> > >> > > > > Yes in fact the actual capture makes use of
> > View.capturePicture()
> > >> > which
> > >> > > > > actually grabs the entire document, then that can be clipped
> > down
> > >> to
> > >> > > the
> > >> > > > > size/location of a specific element.
> > >> > > > >
> > >> > > > > -Aaron
> > >> > > > >
> > >> > > > >
> > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <
> > >> mmocny@chromium.org>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Aaron,
> > >> > > > > >
> > >> > > > > > I haven't even begun looking at your implementation, but I'm
> > >> just
> > >> > > going
> > >> > > > > to
> > >> > > > > > say it: this is awesome!
> > >> > > > > >
> > >> > > > > > First question: When Capturing a DOM element, can you
> capture
> > >> > 'body'
> > >> > > to
> > >> > > > > > grab it&children for a "full content screenshot", or does it
> > >> have
> > >> > to
> > >> > > > be a
> > >> > > > > > specific single element?
> > >> > > > > >
> > >> > > > > > -Michal
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <
> > >> > > amcharbo@gmail.com
> > >> > > > > > >wrote:
> > >> > > > > >
> > >> > > > > > > Greetings! My name is Aaron Charbonneau, happy to be a
> new
> > >> > member
> > >> > > of
> > >> > > > > the
> > >> > > > > > > mailing list!
> > >> > > > > > >
> > >> > > > > > > I have been developing a sceenshot plugin for Cordova to
> > help
> > >> > > > > facilitate
> > >> > > > > > > automation testing and debugging of Cordova apps, and I
> > would
> > >> > love
> > >> > > > some
> > >> > > > > > > feedback on it. Currently Cordova provides a bunch of
> > native
> > >> > > > functions
> > >> > > > > > > that allow you to do some cool stuff, but not much
> > >> functionality
> > >> > to
> > >> > > > > test
> > >> > > > > > > the apps that make use of them. Being able to take a
> > capture
> > >> of
> > >> > > the
> > >> > > > > > screen
> > >> > > > > > > from within you app is a great way to automate testing or
> > get
> > >> > > > > additional
> > >> > > > > > > information for debugging an issue. Since there is no
> > >> > > > > > > Javascript mechanism for taking screen captures the
> solution
> > >> > would
> > >> > > > have
> > >> > > > > > to
> > >> > > > > > > be native, which fits nicely into the "gap" that
> > >> Cordova/Phonegap
> > >> > > > > > bridges.
> > >> > > > > > > Any medium to large scale app can benefit greatly from
> > >> > automation
> > >> > > > > > testing
> > >> > > > > > > and any app can benefit from an extra debugging tool, and
> > >> that is
> > >> > > > what
> > >> > > > > I
> > >> > > > > > > hope this screenshot plugin can help achieve.
> > >> > > > > > >
> > >> > > > > > > Currently the plugin offers 2 functions:
> > >> > > > > > >
> > >> > > > > > > Capture():
> > >> > > > > > > * Take a capture of the current view, write that capture
> to
> > a
> > >> > .png
> > >> > > > file
> > >> > > > > > > with the specified file name and sub directory of the
> sdcard
> > >> > > > (fallback
> > >> > > > > to
> > >> > > > > > > emulated sdcard in the case there isn't an sdcard mounted)
> > >> > > > > > > * Able to create a sub-screenshot with a specified
> rectangle
> > >> in
> > >> > > order
> > >> > > > > to
> > >> > > > > > > block out ui elements that may be variable, and also save
> > >> space.
> > >> > > > > > > * Can take captures of images/dom elements (including
> > canvas)
> > >> > that
> > >> > > > are
> > >> > > > > > > lager than the actual screen size
> > >> > > > > > >
> > >> > > > > > > CaptureAndCompare():
> > >> > > > > > > * All the functionality of Capture()
> > >> > > > > > > * Perform a comparison between the captured image and a
> > >> baseline
> > >> > > > image
> > >> > > > > > > located at the specified location in either the assets
> > folder
> > >> or
> > >> > > the
> > >> > > > > > > sdcard.
> > >> > > > > > > * User can specify per color channel tolerances as well as
> > >> total
> > >> > > > pixel
> > >> > > > > > > tolerances to avoid false positives for the inevitable
> > >> rendering
> > >> > > > > > > differences across devices.
> > >> > > > > > > * Optionally output a png file that contains the
> differences
> > >> > > between
> > >> > > > > the
> > >> > > > > > > actual and the baseline for debugging/triage purposes, two
> > >> modes:
> > >> > > > > binary
> > >> > > > > > > diff (all failing pixels appear as solid white) or the
> true
> > >> > > > differences
> > >> > > > > > > between pixels.
> > >> > > > > > >
> > >> > > > > > > If you can spare some time, I would love it if you could
> > take
> > >> a
> > >> > > look
> > >> > > > at
> > >> > > > > > the
> > >> > > > > > > api and parameters I have defined to make sure they adhere
> > to
> > >> > > Cordova
> > >> > > > > > > plugin best practices. The most crucial part would be in
> > the
> > >> > > plugin
> > >> > > > > > itself
> > >> > > > > > > at ScreenCapture.java:
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > >> > > > > > > and captureAndCompare() are the two exposed functions)
> > >> > > > > > >
> > >> > > > > > > I'm also interested to know you thoughts on it's usage. I
> > >> > started
> > >> > > > with
> > >> > > > > > the
> > >> > > > > > > mobile-spec testing framework and put some quick rendering
> > >> tests
> > >> > > into
> > >> > > > > the
> > >> > > > > > > Autotest section as a viable use case. In order to get
> the
> > >> > WebView
> > >> > > > > into
> > >> > > > > > > the state I wanted to capture, I had to implement a wait
> > >> timer to
> > >> > > > allow
> > >> > > > > > the
> > >> > > > > > > view to update before taking the capture. Once the wait
> > >> function
> > >> > > > > > > completes, the capture can be taken and everything from
> > there
> > >> is
> > >> > > > > callback
> > >> > > > > > > based. I use the Jasmine waitsFor()/runs() blocks to make
> > the
> > >> > > tests
> > >> > > > > run
> > >> > > > > > > synchronously. For some validation testing I made a
> compare
> > >> > > function
> > >> > > > > in
> > >> > > > > > > Javascript to run against the native compare. Turns out
> > Java
> > >> > runs
> > >> > > > alot
> > >> > > > > > > faster than Javascript (surprise!) thus the native compare
> > is
> > >> > about
> > >> > > > > 2-5x
> > >> > > > > > > faster. All in all the process is fairly quick, the full
> > >> > > > > > captureAndCompare
> > >> > > > > > > with 3 file io's takes about 233ms on a Nexus One, and
> > ~100ms
> > >> on
> > >> > a
> > >> > > > > Nexus
> > >> > > > > > 10
> > >> > > > > > > (256x256 image size).
> > >> > > > > > >
> > >> > > > > > > Anyways here is the usage:
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > >> > > > > > >
> > >> > > > > > > And here's the JS wrapper for the plugin calls:
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > >> > > > > > >
> > >> > > > > > > Thanks for your time, and I look forward to ANY feedback
> > >> positive
> > >> > > or
> > >> > > > > > > negative.
> > >> > > > > > >
> > >> > > > > > > Thanks,
> > >> > > > > > > Aaron
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Braden Shepherdson <br...@chromium.org>.
Jasmine does support asynchronous tests, and waiting an arbitrary period or
until some condition is true. Why does that not work for these tests?
Braden
On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <am...@gmail.com>wrote:
> Quick follow up question for the return values stuff:
> Does it make sense to have my functions called from execute (capture() and
> captureAndCompare()) return an error string if one is encountered and pass
> that back in callbackContext.error, then have execute return true?
>
> Or is there a preferred way to get errors from sub functions back into
> Javascript?
>
> Thanks!
>
>
> On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <amcharbo@gmail.com
> >wrote:
>
> > Thanks Andrew these are great suggestions!
> >
> > About not needing the busy-wait in getScreenBits:
> > I'm not a fan of busy-waiting either.
> > Perhaps it's a side effect of Jasmine requiring tests to be run and
> > evaluated synchronously that I decided to use a busy-wait in
> getScreenBits.
> > If the Runnable calls CallbackContext.success/error once the bits are
> > captured (but not written to file) on the UIThread, the Javascript side
> > will proceed to run before it has an image url/compare result, most
> likely
> > causing that test to fail. As far as I know Jasmine doesn't allow for
> > kicking off a bunch of tests and then gather all the results at the end
> and
> > output pass/fails (each 'expect' function needs to be within an 'it'
> block,
> > essentially that means one test must be evaluated before the next can
> > begin).
> >
> > I can see the usefulness of allowing the capture, file io, and comparison
> > to be able to run asynchronously from the Javascript though. In the case
> > where you have your own test framework that CAN kick off a bunch of tests
> > and just gather the results as they come in and do a batch evaluation at
> > the end, or in the case where you don't want to do any evaluation in your
> > app and just want to get captures as fast as you can, asynchronous could
> > increase performance there.
> >
> > I could try to implement it this way too, if you think that's a worth
> > while use case, maybe just provide an 'asynchronous' flag in
> CaptureOptions?
> >
> > For doing the fileIO/compare using cordova.getThreadPool, is there a
> > benefit to doing that if I still plan having the operation block the
> > Javascript? For asynchronous mode I see why that makes sense.
> >
> > For the return values, I'll make those changes right away! Thanks again
> > for providing your insight, hopefully I understood it correctly.
> >
> > -Aaron
> >
> >
> > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <agrieve@chromium.org
> >wrote:
> >
> >> Had a glance at your code. Things you may want to address:
> >>
> >> while(!mUIThreadDone) {}
> >>
> >> You shouldn't need to busy-wait. Pass the CallbackContext to the
> Runnable,
> >> and call success/error on it whenever you're done. No need to have the
> web
> >> thread wait for it. Even better would be to use a second Runnable to do
> >> the
> >> image compare / file IO on a background thread via
> cordova.getThreadPool()
> >>
> >> The return value to execute() should only be false if an invalid command
> >> was passed. It's not meant to indicate the success of the operation. You
> >> should be returning true for it.
> >>
> >> Instead of return "success"/"", it might simplify things a bit to
> return a
> >> boolean of true/false
> >>
> >>
> >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <lorin.beer.dev@gmail.com
> >> >wrote:
> >>
> >> > all the same, very impressive and faster then I expected.
> >> >
> >> > - Lorin
> >> >
> >> >
> >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <
> amcharbo@gmail.com
> >> > >wrote:
> >> >
> >> > > Thanks Lorin.
> >> > >
> >> > > I did a quick test for throughput on an animating canvas. I was
> able
> >> to
> >> > > grab 30 frames within a 5 seconds which equates to around 166ms per
> >> > capture
> >> > > on a Galaxy Nexus. Unfortunately not fast enough to automate
> >> something
> >> > > like video. As with any kind of capturing system, often the
> overhead
> >> of
> >> > > copying bits around is enough to slow down the operation of the app.
> >> > >
> >> > > -Aaron
> >> > >
> >> > >
> >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
> >> lorin.beer.dev@gmail.com
> >> > > >wrote:
> >> > >
> >> > > > Hey Aaron,
> >> > > >
> >> > > > very cool stuff, looking forward to checking it out!
> >> > > >
> >> > > > Question: any performance hit on the app by using Capture? What's
> >> the
> >> > > > expected throughput on images taken in this way (given a
> particular
> >> > > device,
> >> > > > say Galaxy Nexus)?
> >> > > >
> >> > > > - Lorin
> >> > > >
> >> > > >
> >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <
> >> > amcharbo@gmail.com
> >> > > > >wrote:
> >> > > >
> >> > > > > Glad you like it :)
> >> > > > > Yes in fact the actual capture makes use of
> View.capturePicture()
> >> > which
> >> > > > > actually grabs the entire document, then that can be clipped
> down
> >> to
> >> > > the
> >> > > > > size/location of a specific element.
> >> > > > >
> >> > > > > -Aaron
> >> > > > >
> >> > > > >
> >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <
> >> mmocny@chromium.org>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Aaron,
> >> > > > > >
> >> > > > > > I haven't even begun looking at your implementation, but I'm
> >> just
> >> > > going
> >> > > > > to
> >> > > > > > say it: this is awesome!
> >> > > > > >
> >> > > > > > First question: When Capturing a DOM element, can you capture
> >> > 'body'
> >> > > to
> >> > > > > > grab it&children for a "full content screenshot", or does it
> >> have
> >> > to
> >> > > > be a
> >> > > > > > specific single element?
> >> > > > > >
> >> > > > > > -Michal
> >> > > > > >
> >> > > > > >
> >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <
> >> > > amcharbo@gmail.com
> >> > > > > > >wrote:
> >> > > > > >
> >> > > > > > > Greetings! My name is Aaron Charbonneau, happy to be a new
> >> > member
> >> > > of
> >> > > > > the
> >> > > > > > > mailing list!
> >> > > > > > >
> >> > > > > > > I have been developing a sceenshot plugin for Cordova to
> help
> >> > > > > facilitate
> >> > > > > > > automation testing and debugging of Cordova apps, and I
> would
> >> > love
> >> > > > some
> >> > > > > > > feedback on it. Currently Cordova provides a bunch of
> native
> >> > > > functions
> >> > > > > > > that allow you to do some cool stuff, but not much
> >> functionality
> >> > to
> >> > > > > test
> >> > > > > > > the apps that make use of them. Being able to take a
> capture
> >> of
> >> > > the
> >> > > > > > screen
> >> > > > > > > from within you app is a great way to automate testing or
> get
> >> > > > > additional
> >> > > > > > > information for debugging an issue. Since there is no
> >> > > > > > > Javascript mechanism for taking screen captures the solution
> >> > would
> >> > > > have
> >> > > > > > to
> >> > > > > > > be native, which fits nicely into the "gap" that
> >> Cordova/Phonegap
> >> > > > > > bridges.
> >> > > > > > > Any medium to large scale app can benefit greatly from
> >> > automation
> >> > > > > > testing
> >> > > > > > > and any app can benefit from an extra debugging tool, and
> >> that is
> >> > > > what
> >> > > > > I
> >> > > > > > > hope this screenshot plugin can help achieve.
> >> > > > > > >
> >> > > > > > > Currently the plugin offers 2 functions:
> >> > > > > > >
> >> > > > > > > Capture():
> >> > > > > > > * Take a capture of the current view, write that capture to
> a
> >> > .png
> >> > > > file
> >> > > > > > > with the specified file name and sub directory of the sdcard
> >> > > > (fallback
> >> > > > > to
> >> > > > > > > emulated sdcard in the case there isn't an sdcard mounted)
> >> > > > > > > * Able to create a sub-screenshot with a specified rectangle
> >> in
> >> > > order
> >> > > > > to
> >> > > > > > > block out ui elements that may be variable, and also save
> >> space.
> >> > > > > > > * Can take captures of images/dom elements (including
> canvas)
> >> > that
> >> > > > are
> >> > > > > > > lager than the actual screen size
> >> > > > > > >
> >> > > > > > > CaptureAndCompare():
> >> > > > > > > * All the functionality of Capture()
> >> > > > > > > * Perform a comparison between the captured image and a
> >> baseline
> >> > > > image
> >> > > > > > > located at the specified location in either the assets
> folder
> >> or
> >> > > the
> >> > > > > > > sdcard.
> >> > > > > > > * User can specify per color channel tolerances as well as
> >> total
> >> > > > pixel
> >> > > > > > > tolerances to avoid false positives for the inevitable
> >> rendering
> >> > > > > > > differences across devices.
> >> > > > > > > * Optionally output a png file that contains the differences
> >> > > between
> >> > > > > the
> >> > > > > > > actual and the baseline for debugging/triage purposes, two
> >> modes:
> >> > > > > binary
> >> > > > > > > diff (all failing pixels appear as solid white) or the true
> >> > > > differences
> >> > > > > > > between pixels.
> >> > > > > > >
> >> > > > > > > If you can spare some time, I would love it if you could
> take
> >> a
> >> > > look
> >> > > > at
> >> > > > > > the
> >> > > > > > > api and parameters I have defined to make sure they adhere
> to
> >> > > Cordova
> >> > > > > > > plugin best practices. The most crucial part would be in
> the
> >> > > plugin
> >> > > > > > itself
> >> > > > > > > at ScreenCapture.java:
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> >> > > > > > > and captureAndCompare() are the two exposed functions)
> >> > > > > > >
> >> > > > > > > I'm also interested to know you thoughts on it's usage. I
> >> > started
> >> > > > with
> >> > > > > > the
> >> > > > > > > mobile-spec testing framework and put some quick rendering
> >> tests
> >> > > into
> >> > > > > the
> >> > > > > > > Autotest section as a viable use case. In order to get the
> >> > WebView
> >> > > > > into
> >> > > > > > > the state I wanted to capture, I had to implement a wait
> >> timer to
> >> > > > allow
> >> > > > > > the
> >> > > > > > > view to update before taking the capture. Once the wait
> >> function
> >> > > > > > > completes, the capture can be taken and everything from
> there
> >> is
> >> > > > > callback
> >> > > > > > > based. I use the Jasmine waitsFor()/runs() blocks to make
> the
> >> > > tests
> >> > > > > run
> >> > > > > > > synchronously. For some validation testing I made a compare
> >> > > function
> >> > > > > in
> >> > > > > > > Javascript to run against the native compare. Turns out
> Java
> >> > runs
> >> > > > alot
> >> > > > > > > faster than Javascript (surprise!) thus the native compare
> is
> >> > about
> >> > > > > 2-5x
> >> > > > > > > faster. All in all the process is fairly quick, the full
> >> > > > > > captureAndCompare
> >> > > > > > > with 3 file io's takes about 233ms on a Nexus One, and
> ~100ms
> >> on
> >> > a
> >> > > > > Nexus
> >> > > > > > 10
> >> > > > > > > (256x256 image size).
> >> > > > > > >
> >> > > > > > > Anyways here is the usage:
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> >> > > > > > >
> >> > > > > > > And here's the JS wrapper for the plugin calls:
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> >> > > > > > >
> >> > > > > > > Thanks for your time, and I look forward to ANY feedback
> >> positive
> >> > > or
> >> > > > > > > negative.
> >> > > > > > >
> >> > > > > > > Thanks,
> >> > > > > > > Aaron
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Aaron Charbonneau <am...@gmail.com>.
Quick follow up question for the return values stuff:
Does it make sense to have my functions called from execute (capture() and
captureAndCompare()) return an error string if one is encountered and pass
that back in callbackContext.error, then have execute return true?
Or is there a preferred way to get errors from sub functions back into
Javascript?
Thanks!
On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <am...@gmail.com>wrote:
> Thanks Andrew these are great suggestions!
>
> About not needing the busy-wait in getScreenBits:
> I'm not a fan of busy-waiting either.
> Perhaps it's a side effect of Jasmine requiring tests to be run and
> evaluated synchronously that I decided to use a busy-wait in getScreenBits.
> If the Runnable calls CallbackContext.success/error once the bits are
> captured (but not written to file) on the UIThread, the Javascript side
> will proceed to run before it has an image url/compare result, most likely
> causing that test to fail. As far as I know Jasmine doesn't allow for
> kicking off a bunch of tests and then gather all the results at the end and
> output pass/fails (each 'expect' function needs to be within an 'it' block,
> essentially that means one test must be evaluated before the next can
> begin).
>
> I can see the usefulness of allowing the capture, file io, and comparison
> to be able to run asynchronously from the Javascript though. In the case
> where you have your own test framework that CAN kick off a bunch of tests
> and just gather the results as they come in and do a batch evaluation at
> the end, or in the case where you don't want to do any evaluation in your
> app and just want to get captures as fast as you can, asynchronous could
> increase performance there.
>
> I could try to implement it this way too, if you think that's a worth
> while use case, maybe just provide an 'asynchronous' flag in CaptureOptions?
>
> For doing the fileIO/compare using cordova.getThreadPool, is there a
> benefit to doing that if I still plan having the operation block the
> Javascript? For asynchronous mode I see why that makes sense.
>
> For the return values, I'll make those changes right away! Thanks again
> for providing your insight, hopefully I understood it correctly.
>
> -Aaron
>
>
> On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <ag...@chromium.org>wrote:
>
>> Had a glance at your code. Things you may want to address:
>>
>> while(!mUIThreadDone) {}
>>
>> You shouldn't need to busy-wait. Pass the CallbackContext to the Runnable,
>> and call success/error on it whenever you're done. No need to have the web
>> thread wait for it. Even better would be to use a second Runnable to do
>> the
>> image compare / file IO on a background thread via cordova.getThreadPool()
>>
>> The return value to execute() should only be false if an invalid command
>> was passed. It's not meant to indicate the success of the operation. You
>> should be returning true for it.
>>
>> Instead of return "success"/"", it might simplify things a bit to return a
>> boolean of true/false
>>
>>
>> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <lorin.beer.dev@gmail.com
>> >wrote:
>>
>> > all the same, very impressive and faster then I expected.
>> >
>> > - Lorin
>> >
>> >
>> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <amcharbo@gmail.com
>> > >wrote:
>> >
>> > > Thanks Lorin.
>> > >
>> > > I did a quick test for throughput on an animating canvas. I was able
>> to
>> > > grab 30 frames within a 5 seconds which equates to around 166ms per
>> > capture
>> > > on a Galaxy Nexus. Unfortunately not fast enough to automate
>> something
>> > > like video. As with any kind of capturing system, often the overhead
>> of
>> > > copying bits around is enough to slow down the operation of the app.
>> > >
>> > > -Aaron
>> > >
>> > >
>> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
>> lorin.beer.dev@gmail.com
>> > > >wrote:
>> > >
>> > > > Hey Aaron,
>> > > >
>> > > > very cool stuff, looking forward to checking it out!
>> > > >
>> > > > Question: any performance hit on the app by using Capture? What's
>> the
>> > > > expected throughput on images taken in this way (given a particular
>> > > device,
>> > > > say Galaxy Nexus)?
>> > > >
>> > > > - Lorin
>> > > >
>> > > >
>> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <
>> > amcharbo@gmail.com
>> > > > >wrote:
>> > > >
>> > > > > Glad you like it :)
>> > > > > Yes in fact the actual capture makes use of View.capturePicture()
>> > which
>> > > > > actually grabs the entire document, then that can be clipped down
>> to
>> > > the
>> > > > > size/location of a specific element.
>> > > > >
>> > > > > -Aaron
>> > > > >
>> > > > >
>> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <
>> mmocny@chromium.org>
>> > > > > wrote:
>> > > > >
>> > > > > > Aaron,
>> > > > > >
>> > > > > > I haven't even begun looking at your implementation, but I'm
>> just
>> > > going
>> > > > > to
>> > > > > > say it: this is awesome!
>> > > > > >
>> > > > > > First question: When Capturing a DOM element, can you capture
>> > 'body'
>> > > to
>> > > > > > grab it&children for a "full content screenshot", or does it
>> have
>> > to
>> > > > be a
>> > > > > > specific single element?
>> > > > > >
>> > > > > > -Michal
>> > > > > >
>> > > > > >
>> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <
>> > > amcharbo@gmail.com
>> > > > > > >wrote:
>> > > > > >
>> > > > > > > Greetings! My name is Aaron Charbonneau, happy to be a new
>> > member
>> > > of
>> > > > > the
>> > > > > > > mailing list!
>> > > > > > >
>> > > > > > > I have been developing a sceenshot plugin for Cordova to help
>> > > > > facilitate
>> > > > > > > automation testing and debugging of Cordova apps, and I would
>> > love
>> > > > some
>> > > > > > > feedback on it. Currently Cordova provides a bunch of native
>> > > > functions
>> > > > > > > that allow you to do some cool stuff, but not much
>> functionality
>> > to
>> > > > > test
>> > > > > > > the apps that make use of them. Being able to take a capture
>> of
>> > > the
>> > > > > > screen
>> > > > > > > from within you app is a great way to automate testing or get
>> > > > > additional
>> > > > > > > information for debugging an issue. Since there is no
>> > > > > > > Javascript mechanism for taking screen captures the solution
>> > would
>> > > > have
>> > > > > > to
>> > > > > > > be native, which fits nicely into the "gap" that
>> Cordova/Phonegap
>> > > > > > bridges.
>> > > > > > > Any medium to large scale app can benefit greatly from
>> > automation
>> > > > > > testing
>> > > > > > > and any app can benefit from an extra debugging tool, and
>> that is
>> > > > what
>> > > > > I
>> > > > > > > hope this screenshot plugin can help achieve.
>> > > > > > >
>> > > > > > > Currently the plugin offers 2 functions:
>> > > > > > >
>> > > > > > > Capture():
>> > > > > > > * Take a capture of the current view, write that capture to a
>> > .png
>> > > > file
>> > > > > > > with the specified file name and sub directory of the sdcard
>> > > > (fallback
>> > > > > to
>> > > > > > > emulated sdcard in the case there isn't an sdcard mounted)
>> > > > > > > * Able to create a sub-screenshot with a specified rectangle
>> in
>> > > order
>> > > > > to
>> > > > > > > block out ui elements that may be variable, and also save
>> space.
>> > > > > > > * Can take captures of images/dom elements (including canvas)
>> > that
>> > > > are
>> > > > > > > lager than the actual screen size
>> > > > > > >
>> > > > > > > CaptureAndCompare():
>> > > > > > > * All the functionality of Capture()
>> > > > > > > * Perform a comparison between the captured image and a
>> baseline
>> > > > image
>> > > > > > > located at the specified location in either the assets folder
>> or
>> > > the
>> > > > > > > sdcard.
>> > > > > > > * User can specify per color channel tolerances as well as
>> total
>> > > > pixel
>> > > > > > > tolerances to avoid false positives for the inevitable
>> rendering
>> > > > > > > differences across devices.
>> > > > > > > * Optionally output a png file that contains the differences
>> > > between
>> > > > > the
>> > > > > > > actual and the baseline for debugging/triage purposes, two
>> modes:
>> > > > > binary
>> > > > > > > diff (all failing pixels appear as solid white) or the true
>> > > > differences
>> > > > > > > between pixels.
>> > > > > > >
>> > > > > > > If you can spare some time, I would love it if you could take
>> a
>> > > look
>> > > > at
>> > > > > > the
>> > > > > > > api and parameters I have defined to make sure they adhere to
>> > > Cordova
>> > > > > > > plugin best practices. The most crucial part would be in the
>> > > plugin
>> > > > > > itself
>> > > > > > > at ScreenCapture.java:
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
>> > > > > > > and captureAndCompare() are the two exposed functions)
>> > > > > > >
>> > > > > > > I'm also interested to know you thoughts on it's usage. I
>> > started
>> > > > with
>> > > > > > the
>> > > > > > > mobile-spec testing framework and put some quick rendering
>> tests
>> > > into
>> > > > > the
>> > > > > > > Autotest section as a viable use case. In order to get the
>> > WebView
>> > > > > into
>> > > > > > > the state I wanted to capture, I had to implement a wait
>> timer to
>> > > > allow
>> > > > > > the
>> > > > > > > view to update before taking the capture. Once the wait
>> function
>> > > > > > > completes, the capture can be taken and everything from there
>> is
>> > > > > callback
>> > > > > > > based. I use the Jasmine waitsFor()/runs() blocks to make the
>> > > tests
>> > > > > run
>> > > > > > > synchronously. For some validation testing I made a compare
>> > > function
>> > > > > in
>> > > > > > > Javascript to run against the native compare. Turns out Java
>> > runs
>> > > > alot
>> > > > > > > faster than Javascript (surprise!) thus the native compare is
>> > about
>> > > > > 2-5x
>> > > > > > > faster. All in all the process is fairly quick, the full
>> > > > > > captureAndCompare
>> > > > > > > with 3 file io's takes about 233ms on a Nexus One, and ~100ms
>> on
>> > a
>> > > > > Nexus
>> > > > > > 10
>> > > > > > > (256x256 image size).
>> > > > > > >
>> > > > > > > Anyways here is the usage:
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
>> > > > > > >
>> > > > > > > And here's the JS wrapper for the plugin calls:
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
>> > > > > > >
>> > > > > > > Thanks for your time, and I look forward to ANY feedback
>> positive
>> > > or
>> > > > > > > negative.
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > > Aaron
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: Cordova Screen Capture Plugin for Android
Posted by Aaron Charbonneau <am...@gmail.com>.
Thanks Andrew these are great suggestions!
About not needing the busy-wait in getScreenBits:
I'm not a fan of busy-waiting either.
Perhaps it's a side effect of Jasmine requiring tests to be run and
evaluated synchronously that I decided to use a busy-wait in getScreenBits.
If the Runnable calls CallbackContext.success/error once the bits are
captured (but not written to file) on the UIThread, the Javascript side
will proceed to run before it has an image url/compare result, most likely
causing that test to fail. As far as I know Jasmine doesn't allow for
kicking off a bunch of tests and then gather all the results at the end and
output pass/fails (each 'expect' function needs to be within an 'it' block,
essentially that means one test must be evaluated before the next can
begin).
I can see the usefulness of allowing the capture, file io, and comparison
to be able to run asynchronously from the Javascript though. In the case
where you have your own test framework that CAN kick off a bunch of tests
and just gather the results as they come in and do a batch evaluation at
the end, or in the case where you don't want to do any evaluation in your
app and just want to get captures as fast as you can, asynchronous could
increase performance there.
I could try to implement it this way too, if you think that's a worth while
use case, maybe just provide an 'asynchronous' flag in CaptureOptions?
For doing the fileIO/compare using cordova.getThreadPool, is there a
benefit to doing that if I still plan having the operation block the
Javascript? For asynchronous mode I see why that makes sense.
For the return values, I'll make those changes right away! Thanks again
for providing your insight, hopefully I understood it correctly.
-Aaron
On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <ag...@chromium.org> wrote:
> Had a glance at your code. Things you may want to address:
>
> while(!mUIThreadDone) {}
>
> You shouldn't need to busy-wait. Pass the CallbackContext to the Runnable,
> and call success/error on it whenever you're done. No need to have the web
> thread wait for it. Even better would be to use a second Runnable to do the
> image compare / file IO on a background thread via cordova.getThreadPool()
>
> The return value to execute() should only be false if an invalid command
> was passed. It's not meant to indicate the success of the operation. You
> should be returning true for it.
>
> Instead of return "success"/"", it might simplify things a bit to return a
> boolean of true/false
>
>
> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <lorin.beer.dev@gmail.com
> >wrote:
>
> > all the same, very impressive and faster then I expected.
> >
> > - Lorin
> >
> >
> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <amcharbo@gmail.com
> > >wrote:
> >
> > > Thanks Lorin.
> > >
> > > I did a quick test for throughput on an animating canvas. I was able
> to
> > > grab 30 frames within a 5 seconds which equates to around 166ms per
> > capture
> > > on a Galaxy Nexus. Unfortunately not fast enough to automate something
> > > like video. As with any kind of capturing system, often the overhead
> of
> > > copying bits around is enough to slow down the operation of the app.
> > >
> > > -Aaron
> > >
> > >
> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <lorin.beer.dev@gmail.com
> > > >wrote:
> > >
> > > > Hey Aaron,
> > > >
> > > > very cool stuff, looking forward to checking it out!
> > > >
> > > > Question: any performance hit on the app by using Capture? What's the
> > > > expected throughput on images taken in this way (given a particular
> > > device,
> > > > say Galaxy Nexus)?
> > > >
> > > > - Lorin
> > > >
> > > >
> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <
> > amcharbo@gmail.com
> > > > >wrote:
> > > >
> > > > > Glad you like it :)
> > > > > Yes in fact the actual capture makes use of View.capturePicture()
> > which
> > > > > actually grabs the entire document, then that can be clipped down
> to
> > > the
> > > > > size/location of a specific element.
> > > > >
> > > > > -Aaron
> > > > >
> > > > >
> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <
> mmocny@chromium.org>
> > > > > wrote:
> > > > >
> > > > > > Aaron,
> > > > > >
> > > > > > I haven't even begun looking at your implementation, but I'm just
> > > going
> > > > > to
> > > > > > say it: this is awesome!
> > > > > >
> > > > > > First question: When Capturing a DOM element, can you capture
> > 'body'
> > > to
> > > > > > grab it&children for a "full content screenshot", or does it have
> > to
> > > > be a
> > > > > > specific single element?
> > > > > >
> > > > > > -Michal
> > > > > >
> > > > > >
> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <
> > > amcharbo@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > Greetings! My name is Aaron Charbonneau, happy to be a new
> > member
> > > of
> > > > > the
> > > > > > > mailing list!
> > > > > > >
> > > > > > > I have been developing a sceenshot plugin for Cordova to help
> > > > > facilitate
> > > > > > > automation testing and debugging of Cordova apps, and I would
> > love
> > > > some
> > > > > > > feedback on it. Currently Cordova provides a bunch of native
> > > > functions
> > > > > > > that allow you to do some cool stuff, but not much
> functionality
> > to
> > > > > test
> > > > > > > the apps that make use of them. Being able to take a capture
> of
> > > the
> > > > > > screen
> > > > > > > from within you app is a great way to automate testing or get
> > > > > additional
> > > > > > > information for debugging an issue. Since there is no
> > > > > > > Javascript mechanism for taking screen captures the solution
> > would
> > > > have
> > > > > > to
> > > > > > > be native, which fits nicely into the "gap" that
> Cordova/Phonegap
> > > > > > bridges.
> > > > > > > Any medium to large scale app can benefit greatly from
> > automation
> > > > > > testing
> > > > > > > and any app can benefit from an extra debugging tool, and that
> is
> > > > what
> > > > > I
> > > > > > > hope this screenshot plugin can help achieve.
> > > > > > >
> > > > > > > Currently the plugin offers 2 functions:
> > > > > > >
> > > > > > > Capture():
> > > > > > > * Take a capture of the current view, write that capture to a
> > .png
> > > > file
> > > > > > > with the specified file name and sub directory of the sdcard
> > > > (fallback
> > > > > to
> > > > > > > emulated sdcard in the case there isn't an sdcard mounted)
> > > > > > > * Able to create a sub-screenshot with a specified rectangle in
> > > order
> > > > > to
> > > > > > > block out ui elements that may be variable, and also save
> space.
> > > > > > > * Can take captures of images/dom elements (including canvas)
> > that
> > > > are
> > > > > > > lager than the actual screen size
> > > > > > >
> > > > > > > CaptureAndCompare():
> > > > > > > * All the functionality of Capture()
> > > > > > > * Perform a comparison between the captured image and a
> baseline
> > > > image
> > > > > > > located at the specified location in either the assets folder
> or
> > > the
> > > > > > > sdcard.
> > > > > > > * User can specify per color channel tolerances as well as
> total
> > > > pixel
> > > > > > > tolerances to avoid false positives for the inevitable
> rendering
> > > > > > > differences across devices.
> > > > > > > * Optionally output a png file that contains the differences
> > > between
> > > > > the
> > > > > > > actual and the baseline for debugging/triage purposes, two
> modes:
> > > > > binary
> > > > > > > diff (all failing pixels appear as solid white) or the true
> > > > differences
> > > > > > > between pixels.
> > > > > > >
> > > > > > > If you can spare some time, I would love it if you could take a
> > > look
> > > > at
> > > > > > the
> > > > > > > api and parameters I have defined to make sure they adhere to
> > > Cordova
> > > > > > > plugin best practices. The most crucial part would be in the
> > > plugin
> > > > > > itself
> > > > > > > at ScreenCapture.java:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > > > > > and captureAndCompare() are the two exposed functions)
> > > > > > >
> > > > > > > I'm also interested to know you thoughts on it's usage. I
> > started
> > > > with
> > > > > > the
> > > > > > > mobile-spec testing framework and put some quick rendering
> tests
> > > into
> > > > > the
> > > > > > > Autotest section as a viable use case. In order to get the
> > WebView
> > > > > into
> > > > > > > the state I wanted to capture, I had to implement a wait timer
> to
> > > > allow
> > > > > > the
> > > > > > > view to update before taking the capture. Once the wait
> function
> > > > > > > completes, the capture can be taken and everything from there
> is
> > > > > callback
> > > > > > > based. I use the Jasmine waitsFor()/runs() blocks to make the
> > > tests
> > > > > run
> > > > > > > synchronously. For some validation testing I made a compare
> > > function
> > > > > in
> > > > > > > Javascript to run against the native compare. Turns out Java
> > runs
> > > > alot
> > > > > > > faster than Javascript (surprise!) thus the native compare is
> > about
> > > > > 2-5x
> > > > > > > faster. All in all the process is fairly quick, the full
> > > > > > captureAndCompare
> > > > > > > with 3 file io's takes about 233ms on a Nexus One, and ~100ms
> on
> > a
> > > > > Nexus
> > > > > > 10
> > > > > > > (256x256 image size).
> > > > > > >
> > > > > > > Anyways here is the usage:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > > > > > >
> > > > > > > And here's the JS wrapper for the plugin calls:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > > > > > >
> > > > > > > Thanks for your time, and I look forward to ANY feedback
> positive
> > > or
> > > > > > > negative.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Aaron
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Andrew Grieve <ag...@chromium.org>.
Had a glance at your code. Things you may want to address:
while(!mUIThreadDone) {}
You shouldn't need to busy-wait. Pass the CallbackContext to the Runnable,
and call success/error on it whenever you're done. No need to have the web
thread wait for it. Even better would be to use a second Runnable to do the
image compare / file IO on a background thread via cordova.getThreadPool()
The return value to execute() should only be false if an invalid command
was passed. It's not meant to indicate the success of the operation. You
should be returning true for it.
Instead of return "success"/"", it might simplify things a bit to return a
boolean of true/false
On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <lo...@gmail.com>wrote:
> all the same, very impressive and faster then I expected.
>
> - Lorin
>
>
> On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <amcharbo@gmail.com
> >wrote:
>
> > Thanks Lorin.
> >
> > I did a quick test for throughput on an animating canvas. I was able to
> > grab 30 frames within a 5 seconds which equates to around 166ms per
> capture
> > on a Galaxy Nexus. Unfortunately not fast enough to automate something
> > like video. As with any kind of capturing system, often the overhead of
> > copying bits around is enough to slow down the operation of the app.
> >
> > -Aaron
> >
> >
> > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <lorin.beer.dev@gmail.com
> > >wrote:
> >
> > > Hey Aaron,
> > >
> > > very cool stuff, looking forward to checking it out!
> > >
> > > Question: any performance hit on the app by using Capture? What's the
> > > expected throughput on images taken in this way (given a particular
> > device,
> > > say Galaxy Nexus)?
> > >
> > > - Lorin
> > >
> > >
> > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <
> amcharbo@gmail.com
> > > >wrote:
> > >
> > > > Glad you like it :)
> > > > Yes in fact the actual capture makes use of View.capturePicture()
> which
> > > > actually grabs the entire document, then that can be clipped down to
> > the
> > > > size/location of a specific element.
> > > >
> > > > -Aaron
> > > >
> > > >
> > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <mm...@chromium.org>
> > > > wrote:
> > > >
> > > > > Aaron,
> > > > >
> > > > > I haven't even begun looking at your implementation, but I'm just
> > going
> > > > to
> > > > > say it: this is awesome!
> > > > >
> > > > > First question: When Capturing a DOM element, can you capture
> 'body'
> > to
> > > > > grab it&children for a "full content screenshot", or does it have
> to
> > > be a
> > > > > specific single element?
> > > > >
> > > > > -Michal
> > > > >
> > > > >
> > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <
> > amcharbo@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Greetings! My name is Aaron Charbonneau, happy to be a new
> member
> > of
> > > > the
> > > > > > mailing list!
> > > > > >
> > > > > > I have been developing a sceenshot plugin for Cordova to help
> > > > facilitate
> > > > > > automation testing and debugging of Cordova apps, and I would
> love
> > > some
> > > > > > feedback on it. Currently Cordova provides a bunch of native
> > > functions
> > > > > > that allow you to do some cool stuff, but not much functionality
> to
> > > > test
> > > > > > the apps that make use of them. Being able to take a capture of
> > the
> > > > > screen
> > > > > > from within you app is a great way to automate testing or get
> > > > additional
> > > > > > information for debugging an issue. Since there is no
> > > > > > Javascript mechanism for taking screen captures the solution
> would
> > > have
> > > > > to
> > > > > > be native, which fits nicely into the "gap" that Cordova/Phonegap
> > > > > bridges.
> > > > > > Any medium to large scale app can benefit greatly from
> automation
> > > > > testing
> > > > > > and any app can benefit from an extra debugging tool, and that is
> > > what
> > > > I
> > > > > > hope this screenshot plugin can help achieve.
> > > > > >
> > > > > > Currently the plugin offers 2 functions:
> > > > > >
> > > > > > Capture():
> > > > > > * Take a capture of the current view, write that capture to a
> .png
> > > file
> > > > > > with the specified file name and sub directory of the sdcard
> > > (fallback
> > > > to
> > > > > > emulated sdcard in the case there isn't an sdcard mounted)
> > > > > > * Able to create a sub-screenshot with a specified rectangle in
> > order
> > > > to
> > > > > > block out ui elements that may be variable, and also save space.
> > > > > > * Can take captures of images/dom elements (including canvas)
> that
> > > are
> > > > > > lager than the actual screen size
> > > > > >
> > > > > > CaptureAndCompare():
> > > > > > * All the functionality of Capture()
> > > > > > * Perform a comparison between the captured image and a baseline
> > > image
> > > > > > located at the specified location in either the assets folder or
> > the
> > > > > > sdcard.
> > > > > > * User can specify per color channel tolerances as well as total
> > > pixel
> > > > > > tolerances to avoid false positives for the inevitable rendering
> > > > > > differences across devices.
> > > > > > * Optionally output a png file that contains the differences
> > between
> > > > the
> > > > > > actual and the baseline for debugging/triage purposes, two modes:
> > > > binary
> > > > > > diff (all failing pixels appear as solid white) or the true
> > > differences
> > > > > > between pixels.
> > > > > >
> > > > > > If you can spare some time, I would love it if you could take a
> > look
> > > at
> > > > > the
> > > > > > api and parameters I have defined to make sure they adhere to
> > Cordova
> > > > > > plugin best practices. The most crucial part would be in the
> > plugin
> > > > > itself
> > > > > > at ScreenCapture.java:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > > > > and captureAndCompare() are the two exposed functions)
> > > > > >
> > > > > > I'm also interested to know you thoughts on it's usage. I
> started
> > > with
> > > > > the
> > > > > > mobile-spec testing framework and put some quick rendering tests
> > into
> > > > the
> > > > > > Autotest section as a viable use case. In order to get the
> WebView
> > > > into
> > > > > > the state I wanted to capture, I had to implement a wait timer to
> > > allow
> > > > > the
> > > > > > view to update before taking the capture. Once the wait function
> > > > > > completes, the capture can be taken and everything from there is
> > > > callback
> > > > > > based. I use the Jasmine waitsFor()/runs() blocks to make the
> > tests
> > > > run
> > > > > > synchronously. For some validation testing I made a compare
> > function
> > > > in
> > > > > > Javascript to run against the native compare. Turns out Java
> runs
> > > alot
> > > > > > faster than Javascript (surprise!) thus the native compare is
> about
> > > > 2-5x
> > > > > > faster. All in all the process is fairly quick, the full
> > > > > captureAndCompare
> > > > > > with 3 file io's takes about 233ms on a Nexus One, and ~100ms on
> a
> > > > Nexus
> > > > > 10
> > > > > > (256x256 image size).
> > > > > >
> > > > > > Anyways here is the usage:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > > > > >
> > > > > > And here's the JS wrapper for the plugin calls:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > > > > >
> > > > > > Thanks for your time, and I look forward to ANY feedback positive
> > or
> > > > > > negative.
> > > > > >
> > > > > > Thanks,
> > > > > > Aaron
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Lorin Beer <lo...@gmail.com>.
all the same, very impressive and faster then I expected.
- Lorin
On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <am...@gmail.com>wrote:
> Thanks Lorin.
>
> I did a quick test for throughput on an animating canvas. I was able to
> grab 30 frames within a 5 seconds which equates to around 166ms per capture
> on a Galaxy Nexus. Unfortunately not fast enough to automate something
> like video. As with any kind of capturing system, often the overhead of
> copying bits around is enough to slow down the operation of the app.
>
> -Aaron
>
>
> On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <lorin.beer.dev@gmail.com
> >wrote:
>
> > Hey Aaron,
> >
> > very cool stuff, looking forward to checking it out!
> >
> > Question: any performance hit on the app by using Capture? What's the
> > expected throughput on images taken in this way (given a particular
> device,
> > say Galaxy Nexus)?
> >
> > - Lorin
> >
> >
> > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <amcharbo@gmail.com
> > >wrote:
> >
> > > Glad you like it :)
> > > Yes in fact the actual capture makes use of View.capturePicture() which
> > > actually grabs the entire document, then that can be clipped down to
> the
> > > size/location of a specific element.
> > >
> > > -Aaron
> > >
> > >
> > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <mm...@chromium.org>
> > > wrote:
> > >
> > > > Aaron,
> > > >
> > > > I haven't even begun looking at your implementation, but I'm just
> going
> > > to
> > > > say it: this is awesome!
> > > >
> > > > First question: When Capturing a DOM element, can you capture 'body'
> to
> > > > grab it&children for a "full content screenshot", or does it have to
> > be a
> > > > specific single element?
> > > >
> > > > -Michal
> > > >
> > > >
> > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <
> amcharbo@gmail.com
> > > > >wrote:
> > > >
> > > > > Greetings! My name is Aaron Charbonneau, happy to be a new member
> of
> > > the
> > > > > mailing list!
> > > > >
> > > > > I have been developing a sceenshot plugin for Cordova to help
> > > facilitate
> > > > > automation testing and debugging of Cordova apps, and I would love
> > some
> > > > > feedback on it. Currently Cordova provides a bunch of native
> > functions
> > > > > that allow you to do some cool stuff, but not much functionality to
> > > test
> > > > > the apps that make use of them. Being able to take a capture of
> the
> > > > screen
> > > > > from within you app is a great way to automate testing or get
> > > additional
> > > > > information for debugging an issue. Since there is no
> > > > > Javascript mechanism for taking screen captures the solution would
> > have
> > > > to
> > > > > be native, which fits nicely into the "gap" that Cordova/Phonegap
> > > > bridges.
> > > > > Any medium to large scale app can benefit greatly from automation
> > > > testing
> > > > > and any app can benefit from an extra debugging tool, and that is
> > what
> > > I
> > > > > hope this screenshot plugin can help achieve.
> > > > >
> > > > > Currently the plugin offers 2 functions:
> > > > >
> > > > > Capture():
> > > > > * Take a capture of the current view, write that capture to a .png
> > file
> > > > > with the specified file name and sub directory of the sdcard
> > (fallback
> > > to
> > > > > emulated sdcard in the case there isn't an sdcard mounted)
> > > > > * Able to create a sub-screenshot with a specified rectangle in
> order
> > > to
> > > > > block out ui elements that may be variable, and also save space.
> > > > > * Can take captures of images/dom elements (including canvas) that
> > are
> > > > > lager than the actual screen size
> > > > >
> > > > > CaptureAndCompare():
> > > > > * All the functionality of Capture()
> > > > > * Perform a comparison between the captured image and a baseline
> > image
> > > > > located at the specified location in either the assets folder or
> the
> > > > > sdcard.
> > > > > * User can specify per color channel tolerances as well as total
> > pixel
> > > > > tolerances to avoid false positives for the inevitable rendering
> > > > > differences across devices.
> > > > > * Optionally output a png file that contains the differences
> between
> > > the
> > > > > actual and the baseline for debugging/triage purposes, two modes:
> > > binary
> > > > > diff (all failing pixels appear as solid white) or the true
> > differences
> > > > > between pixels.
> > > > >
> > > > > If you can spare some time, I would love it if you could take a
> look
> > at
> > > > the
> > > > > api and parameters I have defined to make sure they adhere to
> Cordova
> > > > > plugin best practices. The most crucial part would be in the
> plugin
> > > > itself
> > > > > at ScreenCapture.java:
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > > > and captureAndCompare() are the two exposed functions)
> > > > >
> > > > > I'm also interested to know you thoughts on it's usage. I started
> > with
> > > > the
> > > > > mobile-spec testing framework and put some quick rendering tests
> into
> > > the
> > > > > Autotest section as a viable use case. In order to get the WebView
> > > into
> > > > > the state I wanted to capture, I had to implement a wait timer to
> > allow
> > > > the
> > > > > view to update before taking the capture. Once the wait function
> > > > > completes, the capture can be taken and everything from there is
> > > callback
> > > > > based. I use the Jasmine waitsFor()/runs() blocks to make the
> tests
> > > run
> > > > > synchronously. For some validation testing I made a compare
> function
> > > in
> > > > > Javascript to run against the native compare. Turns out Java runs
> > alot
> > > > > faster than Javascript (surprise!) thus the native compare is about
> > > 2-5x
> > > > > faster. All in all the process is fairly quick, the full
> > > > captureAndCompare
> > > > > with 3 file io's takes about 233ms on a Nexus One, and ~100ms on a
> > > Nexus
> > > > 10
> > > > > (256x256 image size).
> > > > >
> > > > > Anyways here is the usage:
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > > > >
> > > > > And here's the JS wrapper for the plugin calls:
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > > > >
> > > > > Thanks for your time, and I look forward to ANY feedback positive
> or
> > > > > negative.
> > > > >
> > > > > Thanks,
> > > > > Aaron
> > > > >
> > > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Aaron Charbonneau <am...@gmail.com>.
Thanks Lorin.
I did a quick test for throughput on an animating canvas. I was able to
grab 30 frames within a 5 seconds which equates to around 166ms per capture
on a Galaxy Nexus. Unfortunately not fast enough to automate something
like video. As with any kind of capturing system, often the overhead of
copying bits around is enough to slow down the operation of the app.
-Aaron
On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <lo...@gmail.com>wrote:
> Hey Aaron,
>
> very cool stuff, looking forward to checking it out!
>
> Question: any performance hit on the app by using Capture? What's the
> expected throughput on images taken in this way (given a particular device,
> say Galaxy Nexus)?
>
> - Lorin
>
>
> On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <amcharbo@gmail.com
> >wrote:
>
> > Glad you like it :)
> > Yes in fact the actual capture makes use of View.capturePicture() which
> > actually grabs the entire document, then that can be clipped down to the
> > size/location of a specific element.
> >
> > -Aaron
> >
> >
> > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <mm...@chromium.org>
> > wrote:
> >
> > > Aaron,
> > >
> > > I haven't even begun looking at your implementation, but I'm just going
> > to
> > > say it: this is awesome!
> > >
> > > First question: When Capturing a DOM element, can you capture 'body' to
> > > grab it&children for a "full content screenshot", or does it have to
> be a
> > > specific single element?
> > >
> > > -Michal
> > >
> > >
> > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <amcharbo@gmail.com
> > > >wrote:
> > >
> > > > Greetings! My name is Aaron Charbonneau, happy to be a new member of
> > the
> > > > mailing list!
> > > >
> > > > I have been developing a sceenshot plugin for Cordova to help
> > facilitate
> > > > automation testing and debugging of Cordova apps, and I would love
> some
> > > > feedback on it. Currently Cordova provides a bunch of native
> functions
> > > > that allow you to do some cool stuff, but not much functionality to
> > test
> > > > the apps that make use of them. Being able to take a capture of the
> > > screen
> > > > from within you app is a great way to automate testing or get
> > additional
> > > > information for debugging an issue. Since there is no
> > > > Javascript mechanism for taking screen captures the solution would
> have
> > > to
> > > > be native, which fits nicely into the "gap" that Cordova/Phonegap
> > > bridges.
> > > > Any medium to large scale app can benefit greatly from automation
> > > testing
> > > > and any app can benefit from an extra debugging tool, and that is
> what
> > I
> > > > hope this screenshot plugin can help achieve.
> > > >
> > > > Currently the plugin offers 2 functions:
> > > >
> > > > Capture():
> > > > * Take a capture of the current view, write that capture to a .png
> file
> > > > with the specified file name and sub directory of the sdcard
> (fallback
> > to
> > > > emulated sdcard in the case there isn't an sdcard mounted)
> > > > * Able to create a sub-screenshot with a specified rectangle in order
> > to
> > > > block out ui elements that may be variable, and also save space.
> > > > * Can take captures of images/dom elements (including canvas) that
> are
> > > > lager than the actual screen size
> > > >
> > > > CaptureAndCompare():
> > > > * All the functionality of Capture()
> > > > * Perform a comparison between the captured image and a baseline
> image
> > > > located at the specified location in either the assets folder or the
> > > > sdcard.
> > > > * User can specify per color channel tolerances as well as total
> pixel
> > > > tolerances to avoid false positives for the inevitable rendering
> > > > differences across devices.
> > > > * Optionally output a png file that contains the differences between
> > the
> > > > actual and the baseline for debugging/triage purposes, two modes:
> > binary
> > > > diff (all failing pixels appear as solid white) or the true
> differences
> > > > between pixels.
> > > >
> > > > If you can spare some time, I would love it if you could take a look
> at
> > > the
> > > > api and parameters I have defined to make sure they adhere to Cordova
> > > > plugin best practices. The most crucial part would be in the plugin
> > > itself
> > > > at ScreenCapture.java:
> > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > > and captureAndCompare() are the two exposed functions)
> > > >
> > > > I'm also interested to know you thoughts on it's usage. I started
> with
> > > the
> > > > mobile-spec testing framework and put some quick rendering tests into
> > the
> > > > Autotest section as a viable use case. In order to get the WebView
> > into
> > > > the state I wanted to capture, I had to implement a wait timer to
> allow
> > > the
> > > > view to update before taking the capture. Once the wait function
> > > > completes, the capture can be taken and everything from there is
> > callback
> > > > based. I use the Jasmine waitsFor()/runs() blocks to make the tests
> > run
> > > > synchronously. For some validation testing I made a compare function
> > in
> > > > Javascript to run against the native compare. Turns out Java runs
> alot
> > > > faster than Javascript (surprise!) thus the native compare is about
> > 2-5x
> > > > faster. All in all the process is fairly quick, the full
> > > captureAndCompare
> > > > with 3 file io's takes about 233ms on a Nexus One, and ~100ms on a
> > Nexus
> > > 10
> > > > (256x256 image size).
> > > >
> > > > Anyways here is the usage:
> > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > > >
> > > > And here's the JS wrapper for the plugin calls:
> > > >
> > > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > > >
> > > > Thanks for your time, and I look forward to ANY feedback positive or
> > > > negative.
> > > >
> > > > Thanks,
> > > > Aaron
> > > >
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Lorin Beer <lo...@gmail.com>.
Hey Aaron,
very cool stuff, looking forward to checking it out!
Question: any performance hit on the app by using Capture? What's the
expected throughput on images taken in this way (given a particular device,
say Galaxy Nexus)?
- Lorin
On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <am...@gmail.com>wrote:
> Glad you like it :)
> Yes in fact the actual capture makes use of View.capturePicture() which
> actually grabs the entire document, then that can be clipped down to the
> size/location of a specific element.
>
> -Aaron
>
>
> On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <mm...@chromium.org>
> wrote:
>
> > Aaron,
> >
> > I haven't even begun looking at your implementation, but I'm just going
> to
> > say it: this is awesome!
> >
> > First question: When Capturing a DOM element, can you capture 'body' to
> > grab it&children for a "full content screenshot", or does it have to be a
> > specific single element?
> >
> > -Michal
> >
> >
> > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <amcharbo@gmail.com
> > >wrote:
> >
> > > Greetings! My name is Aaron Charbonneau, happy to be a new member of
> the
> > > mailing list!
> > >
> > > I have been developing a sceenshot plugin for Cordova to help
> facilitate
> > > automation testing and debugging of Cordova apps, and I would love some
> > > feedback on it. Currently Cordova provides a bunch of native functions
> > > that allow you to do some cool stuff, but not much functionality to
> test
> > > the apps that make use of them. Being able to take a capture of the
> > screen
> > > from within you app is a great way to automate testing or get
> additional
> > > information for debugging an issue. Since there is no
> > > Javascript mechanism for taking screen captures the solution would have
> > to
> > > be native, which fits nicely into the "gap" that Cordova/Phonegap
> > bridges.
> > > Any medium to large scale app can benefit greatly from automation
> > testing
> > > and any app can benefit from an extra debugging tool, and that is what
> I
> > > hope this screenshot plugin can help achieve.
> > >
> > > Currently the plugin offers 2 functions:
> > >
> > > Capture():
> > > * Take a capture of the current view, write that capture to a .png file
> > > with the specified file name and sub directory of the sdcard (fallback
> to
> > > emulated sdcard in the case there isn't an sdcard mounted)
> > > * Able to create a sub-screenshot with a specified rectangle in order
> to
> > > block out ui elements that may be variable, and also save space.
> > > * Can take captures of images/dom elements (including canvas) that are
> > > lager than the actual screen size
> > >
> > > CaptureAndCompare():
> > > * All the functionality of Capture()
> > > * Perform a comparison between the captured image and a baseline image
> > > located at the specified location in either the assets folder or the
> > > sdcard.
> > > * User can specify per color channel tolerances as well as total pixel
> > > tolerances to avoid false positives for the inevitable rendering
> > > differences across devices.
> > > * Optionally output a png file that contains the differences between
> the
> > > actual and the baseline for debugging/triage purposes, two modes:
> binary
> > > diff (all failing pixels appear as solid white) or the true differences
> > > between pixels.
> > >
> > > If you can spare some time, I would love it if you could take a look at
> > the
> > > api and parameters I have defined to make sure they adhere to Cordova
> > > plugin best practices. The most crucial part would be in the plugin
> > itself
> > > at ScreenCapture.java:
> > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > > and captureAndCompare() are the two exposed functions)
> > >
> > > I'm also interested to know you thoughts on it's usage. I started with
> > the
> > > mobile-spec testing framework and put some quick rendering tests into
> the
> > > Autotest section as a viable use case. In order to get the WebView
> into
> > > the state I wanted to capture, I had to implement a wait timer to allow
> > the
> > > view to update before taking the capture. Once the wait function
> > > completes, the capture can be taken and everything from there is
> callback
> > > based. I use the Jasmine waitsFor()/runs() blocks to make the tests
> run
> > > synchronously. For some validation testing I made a compare function
> in
> > > Javascript to run against the native compare. Turns out Java runs alot
> > > faster than Javascript (surprise!) thus the native compare is about
> 2-5x
> > > faster. All in all the process is fairly quick, the full
> > captureAndCompare
> > > with 3 file io's takes about 233ms on a Nexus One, and ~100ms on a
> Nexus
> > 10
> > > (256x256 image size).
> > >
> > > Anyways here is the usage:
> > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > >
> > > And here's the JS wrapper for the plugin calls:
> > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > >
> > > Thanks for your time, and I look forward to ANY feedback positive or
> > > negative.
> > >
> > > Thanks,
> > > Aaron
> > >
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Aaron Charbonneau <am...@gmail.com>.
Glad you like it :)
Yes in fact the actual capture makes use of View.capturePicture() which
actually grabs the entire document, then that can be clipped down to the
size/location of a specific element.
-Aaron
On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <mm...@chromium.org> wrote:
> Aaron,
>
> I haven't even begun looking at your implementation, but I'm just going to
> say it: this is awesome!
>
> First question: When Capturing a DOM element, can you capture 'body' to
> grab it&children for a "full content screenshot", or does it have to be a
> specific single element?
>
> -Michal
>
>
> On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <amcharbo@gmail.com
> >wrote:
>
> > Greetings! My name is Aaron Charbonneau, happy to be a new member of the
> > mailing list!
> >
> > I have been developing a sceenshot plugin for Cordova to help facilitate
> > automation testing and debugging of Cordova apps, and I would love some
> > feedback on it. Currently Cordova provides a bunch of native functions
> > that allow you to do some cool stuff, but not much functionality to test
> > the apps that make use of them. Being able to take a capture of the
> screen
> > from within you app is a great way to automate testing or get additional
> > information for debugging an issue. Since there is no
> > Javascript mechanism for taking screen captures the solution would have
> to
> > be native, which fits nicely into the "gap" that Cordova/Phonegap
> bridges.
> > Any medium to large scale app can benefit greatly from automation
> testing
> > and any app can benefit from an extra debugging tool, and that is what I
> > hope this screenshot plugin can help achieve.
> >
> > Currently the plugin offers 2 functions:
> >
> > Capture():
> > * Take a capture of the current view, write that capture to a .png file
> > with the specified file name and sub directory of the sdcard (fallback to
> > emulated sdcard in the case there isn't an sdcard mounted)
> > * Able to create a sub-screenshot with a specified rectangle in order to
> > block out ui elements that may be variable, and also save space.
> > * Can take captures of images/dom elements (including canvas) that are
> > lager than the actual screen size
> >
> > CaptureAndCompare():
> > * All the functionality of Capture()
> > * Perform a comparison between the captured image and a baseline image
> > located at the specified location in either the assets folder or the
> > sdcard.
> > * User can specify per color channel tolerances as well as total pixel
> > tolerances to avoid false positives for the inevitable rendering
> > differences across devices.
> > * Optionally output a png file that contains the differences between the
> > actual and the baseline for debugging/triage purposes, two modes: binary
> > diff (all failing pixels appear as solid white) or the true differences
> > between pixels.
> >
> > If you can spare some time, I would love it if you could take a look at
> the
> > api and parameters I have defined to make sure they adhere to Cordova
> > plugin best practices. The most crucial part would be in the plugin
> itself
> > at ScreenCapture.java:
> >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > and captureAndCompare() are the two exposed functions)
> >
> > I'm also interested to know you thoughts on it's usage. I started with
> the
> > mobile-spec testing framework and put some quick rendering tests into the
> > Autotest section as a viable use case. In order to get the WebView into
> > the state I wanted to capture, I had to implement a wait timer to allow
> the
> > view to update before taking the capture. Once the wait function
> > completes, the capture can be taken and everything from there is callback
> > based. I use the Jasmine waitsFor()/runs() blocks to make the tests run
> > synchronously. For some validation testing I made a compare function in
> > Javascript to run against the native compare. Turns out Java runs alot
> > faster than Javascript (surprise!) thus the native compare is about 2-5x
> > faster. All in all the process is fairly quick, the full
> captureAndCompare
> > with 3 file io's takes about 233ms on a Nexus One, and ~100ms on a Nexus
> 10
> > (256x256 image size).
> >
> > Anyways here is the usage:
> >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> >
> > And here's the JS wrapper for the plugin calls:
> >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> >
> > Thanks for your time, and I look forward to ANY feedback positive or
> > negative.
> >
> > Thanks,
> > Aaron
> >
>
Re: Cordova Screen Capture Plugin for Android
Posted by Michal Mocny <mm...@chromium.org>.
Aaron,
I haven't even begun looking at your implementation, but I'm just going to
say it: this is awesome!
First question: When Capturing a DOM element, can you capture 'body' to
grab it&children for a "full content screenshot", or does it have to be a
specific single element?
-Michal
On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <am...@gmail.com>wrote:
> Greetings! My name is Aaron Charbonneau, happy to be a new member of the
> mailing list!
>
> I have been developing a sceenshot plugin for Cordova to help facilitate
> automation testing and debugging of Cordova apps, and I would love some
> feedback on it. Currently Cordova provides a bunch of native functions
> that allow you to do some cool stuff, but not much functionality to test
> the apps that make use of them. Being able to take a capture of the screen
> from within you app is a great way to automate testing or get additional
> information for debugging an issue. Since there is no
> Javascript mechanism for taking screen captures the solution would have to
> be native, which fits nicely into the "gap" that Cordova/Phonegap bridges.
> Any medium to large scale app can benefit greatly from automation testing
> and any app can benefit from an extra debugging tool, and that is what I
> hope this screenshot plugin can help achieve.
>
> Currently the plugin offers 2 functions:
>
> Capture():
> * Take a capture of the current view, write that capture to a .png file
> with the specified file name and sub directory of the sdcard (fallback to
> emulated sdcard in the case there isn't an sdcard mounted)
> * Able to create a sub-screenshot with a specified rectangle in order to
> block out ui elements that may be variable, and also save space.
> * Can take captures of images/dom elements (including canvas) that are
> lager than the actual screen size
>
> CaptureAndCompare():
> * All the functionality of Capture()
> * Perform a comparison between the captured image and a baseline image
> located at the specified location in either the assets folder or the
> sdcard.
> * User can specify per color channel tolerances as well as total pixel
> tolerances to avoid false positives for the inevitable rendering
> differences across devices.
> * Optionally output a png file that contains the differences between the
> actual and the baseline for debugging/triage purposes, two modes: binary
> diff (all failing pixels appear as solid white) or the true differences
> between pixels.
>
> If you can spare some time, I would love it if you could take a look at the
> api and parameters I have defined to make sure they adhere to Cordova
> plugin best practices. The most crucial part would be in the plugin itself
> at ScreenCapture.java:
>
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> and captureAndCompare() are the two exposed functions)
>
> I'm also interested to know you thoughts on it's usage. I started with the
> mobile-spec testing framework and put some quick rendering tests into the
> Autotest section as a viable use case. In order to get the WebView into
> the state I wanted to capture, I had to implement a wait timer to allow the
> view to update before taking the capture. Once the wait function
> completes, the capture can be taken and everything from there is callback
> based. I use the Jasmine waitsFor()/runs() blocks to make the tests run
> synchronously. For some validation testing I made a compare function in
> Javascript to run against the native compare. Turns out Java runs alot
> faster than Javascript (surprise!) thus the native compare is about 2-5x
> faster. All in all the process is fairly quick, the full captureAndCompare
> with 3 file io's takes about 233ms on a Nexus One, and ~100ms on a Nexus 10
> (256x256 image size).
>
> Anyways here is the usage:
>
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
>
> And here's the JS wrapper for the plugin calls:
>
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
>
> Thanks for your time, and I look forward to ANY feedback positive or
> negative.
>
> Thanks,
> Aaron
>