You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cordova.apache.org by Filip Maj <ma...@gmail.com> on 2016/12/05 22:33:50 UTC
An idea for manually testing plugins
Hi, it's me again!
How I'd like to contribute to Cordova is to help automate the stuff
that saves committers having to take the manual time to do themselves.
I think a good first goal would be to help automate as much of
platform release testing as possible [1]. The autotests seem to be
handled relatively well by the CI system so far (if anyone feels this
is not true, please speak up! I want to hear about what works and what
does not). The other part of mobile-spec-based platform release
testing is to "run the manual tests". After reading the
cordova-plugin-test-framework README [2], I thought this would be a
good place to give plugin developers some better tools to deal with
testing complex UI interactions that the autotests can't handle on
their own. I was thinking appium [3] would be a good tool to
complement that in this case. It gives us UI hooks into both web and
natives contexts within hybrid applications, plus it also allows us to
inject JavaScript into the web context. Wondering what others think?
I could then foresee, with a little bit of scaffolding, a way to
string plugins' appium tests together to fully automate the 'manual'
testing of plugin tests during platform release testing.
Let me know what y'all think!
[1] https://github.com/apache/cordova-coho/blob/master/docs/platforms-release-process.md#what-to-test
[2] https://github.com/apache/cordova-plugin-test-framework#defining-manual-tests
[3] appium.io
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
For additional commands, e-mail: dev-help@cordova.apache.org
Re: An idea for manually testing plugins
Posted by Filip Maj <ma...@gmail.com>.
As Alex Sorokin kindly pointed out to me on Slack, we have Appium UI
automation already existing for the contacts [1] and camera [2]
plugins. Our CI system also executes on this to automate the UI
end-to-end and guard against further errors [3]. The CI tosses this
job over to Sauce Labs to do the heavy lifting: Sauce runs the test
[4] from the example from [3], and, based on the logs, looks to
execute all that fun web + native context UI automation -
unfortunately Sauce only stores assets around for a month, so we can't
see the video or screenshots (it's been over a month since the last
contacts PR).
But all that's to say that, with that in place, I think it's mostly a
matter of applying this approach to other plugins. I think the value
of the UI automation-based tests is wayyyy higher than the autotests;
asserting that a JavaScript function exists inside the Cordova webview
is a good sanity check, but asserting that you can pick a contact via
a native UI and have the relevant contact details show up back in the
Cordova webview, in my opinion, is a much more thorough test of the
functions of the plugin APIs.
Lastly, I think documenting how to set up UI automation tests such as
the ones in the camera and contacts plugins in the
cordova-plugin-test-framework readme makes sense.
Let me know what y'all think.
[1] https://github.com/apache/cordova-plugin-contacts/blob/master/appium-tests
[2] https://github.com/apache/cordova-plugin-camera/blob/master/appium-tests
[3] http://cordova-ci.cloudapp.net:8080/view/Pull%20requests/job/cordova-plugin-contacts-pr/lastSuccessfulBuild/PLATFORM=ios/consoleText
[4] https://saucelabs.com/beta/tests/b014505bd8894ff58b383471bdaf4759
On Wed, Dec 7, 2016 at 6:14 AM, Filip Maj <ma...@gmail.com> wrote:
> I am planning on putting a proof of concept together for a few plugins, and
> having something like mobile spec be able to string multiple plugins' UI
> automation tests together and assert pass/failure (via its defineAutoTests /
> defineManualTests-like functions).
>
> I'll post more when I have something to show :)
>
> The idea would then be to help guide how to do this kind of testing via the
> framework Cordova has built up for plugin testing via the
> cordova-plugin-test-framework plugin.
>
> On Dec 7, 2016 05:56, "julio cesar sanchez" <jc...@gmail.com> wrote:
>>
>> I was the other way around, I got Appium working easily, but I was never
>> able to make calabash work in iOS because it always failed to instrument
>> my
>> app.
>> Both worked fine for Android though.
>>
>>
>> 2016-12-07 12:48 GMT+01:00 Trevor Brindle <ta...@gmail.com>:
>>
>> > As little market share as windows has, it's hard to say how much effort
>> > should be dedicated to additional testing for it. That said, if windows
>> > testing is deemed necessary, I don't think we have much choice.
>> >
>> > Just keep in mind the new windows support and new iOS driver are all
>> > appium
>> > 1.6, only released less than 60 days ago.
>> >
>> > Perhaps a PoC project should be done for picking the appropriate
>> > technology?
>> >
>> > We are midway through a similar PoC at my place of work, I could
>> > volunteer
>> > some time.
>> >
>> > On Tue, Dec 6, 2016 at 10:02 PM Filip Maj <ma...@gmail.com> wrote:
>> >
>> > > Full disclosure: I contribute(d) to appium and worked for Sauce Labs,
>> > >
>> > > so I am pretty biased ;)
>> > >
>> > >
>> > >
>> > > Calabash, unfortunately, is iOS and Android only and does not support
>> > >
>> > > Windows app, whereas Appium does via WinAppDriver [1].
>> > >
>> > >
>> > >
>> > > I agree that appium, at least earlier on, was difficult to set up.
>> > >
>> > > These days it's a simple `npm install`. I think appium has a much more
>> > >
>> > > prolific committership to boot (MSFT contributes to it, for example).
>> > >
>> > > The github network stats for each project back that up as well.
>> > >
>> > >
>> > >
>> > > [1] https://github.com/microsoft/winappdriver
>> > >
>> > >
>> > >
>> > > On Tue, Dec 6, 2016 at 4:02 PM, Trevor Brindle <ta...@gmail.com>
>> > > wrote:
>> > >
>> > > > In my experience with appium, it is troublesome to set up and
>> > > > maintain.
>> > > We
>> > >
>> > > > may investigate other frameworks that have similar functionality. I
>> > have
>> > >
>> > > > had great luck with calabash.
>> > >
>> > > >
>> > >
>> > > >
>> > >
>> > > > On Tue, Dec 6, 2016 at 5:30 PM julio cesar sanchez <
>> > > jcesarmobile@gmail.com>
>> > >
>> > > > wrote:
>> > >
>> > > >
>> > >
>> > > >> +1 to appium
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >> 2016-12-05 23:33 GMT+01:00 Filip Maj <ma...@gmail.com>:
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > > >> > Hi, it's me again!
>> > >
>> > > >>
>> > >
>> > > >> >
>> > >
>> > > >>
>> > >
>> > > >> > How I'd like to contribute to Cordova is to help automate the
>> > > >> > stuff
>> > >
>> > > >>
>> > >
>> > > >> > that saves committers having to take the manual time to do
>> > themselves.
>> > >
>> > > >>
>> > >
>> > > >> > I think a good first goal would be to help automate as much of
>> > >
>> > > >>
>> > >
>> > > >> > platform release testing as possible [1]. The autotests seem to
>> > > >> > be
>> > >
>> > > >>
>> > >
>> > > >> > handled relatively well by the CI system so far (if anyone feels
>> > this
>> > >
>> > > >>
>> > >
>> > > >> > is not true, please speak up! I want to hear about what works and
>> > what
>> > >
>> > > >>
>> > >
>> > > >> > does not). The other part of mobile-spec-based platform release
>> > >
>> > > >>
>> > >
>> > > >> > testing is to "run the manual tests". After reading the
>> > >
>> > > >>
>> > >
>> > > >> > cordova-plugin-test-framework README [2], I thought this would be
>> > > >> > a
>> > >
>> > > >>
>> > >
>> > > >> > good place to give plugin developers some better tools to deal
>> > > >> > with
>> > >
>> > > >>
>> > >
>> > > >> > testing complex UI interactions that the autotests can't handle
>> > > >> > on
>> > >
>> > > >>
>> > >
>> > > >> > their own. I was thinking appium [3] would be a good tool to
>> > >
>> > > >>
>> > >
>> > > >> > complement that in this case. It gives us UI hooks into both web
>> > > >> > and
>> > >
>> > > >>
>> > >
>> > > >> > natives contexts within hybrid applications, plus it also allows
>> > > >> > us
>> > to
>> > >
>> > > >>
>> > >
>> > > >> > inject JavaScript into the web context. Wondering what others
>> > > >> > think?
>> > >
>> > > >>
>> > >
>> > > >> >
>> > >
>> > > >>
>> > >
>> > > >> > I could then foresee, with a little bit of scaffolding, a way to
>> > >
>> > > >>
>> > >
>> > > >> > string plugins' appium tests together to fully automate the
>> > > >> > 'manual'
>> > >
>> > > >>
>> > >
>> > > >> > testing of plugin tests during platform release testing.
>> > >
>> > > >>
>> > >
>> > > >> >
>> > >
>> > > >>
>> > >
>> > > >> > Let me know what y'all think!
>> > >
>> > > >>
>> > >
>> > > >> >
>> > >
>> > > >>
>> > >
>> > > >> > [1] https://github.com/apache/cordova-coho/blob/master/docs/
>> > >
>> > > >>
>> > >
>> > > >> > platforms-release-process.md#what-to-test
>> > >
>> > > >>
>> > >
>> > > >> > [2] https://github.com/apache/cordova-plugin-test-framework#
>> > >
>> > > >>
>> > >
>> > > >> > defining-manual-tests
>> > >
>> > > >>
>> > >
>> > > >> > [3] appium.io
>> > >
>> > > >>
>> > >
>> > > >> >
>> > >
>> > > >>
>> > >
>> > > >> > ------------------------------------------------------------
>> > ---------
>> > >
>> > > >>
>> > >
>> > > >> > To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
>> > >
>> > > >>
>> > >
>> > > >> > For additional commands, e-mail: dev-help@cordova.apache.org
>> > >
>> > > >>
>> > >
>> > > >> >
>> > >
>> > > >>
>> > >
>> > > >> >
>> > >
>> > > >>
>> > >
>> > > >>
>> > >
>> > >
>> > >
>> > > ---------------------------------------------------------------------
>> > >
>> > > To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
>> > >
>> > > For additional commands, e-mail: dev-help@cordova.apache.org
>> > >
>> > >
>> > >
>> > >
>> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
For additional commands, e-mail: dev-help@cordova.apache.org
Re: An idea for manually testing plugins
Posted by Filip Maj <ma...@gmail.com>.
I am planning on putting a proof of concept together for a few plugins, and
having something like mobile spec be able to string multiple plugins' UI
automation tests together and assert pass/failure (via its defineAutoTests
/ defineManualTests-like functions).
I'll post more when I have something to show :)
The idea would then be to help guide how to do this kind of testing via the
framework Cordova has built up for plugin testing via the
cordova-plugin-test-framework plugin.
On Dec 7, 2016 05:56, "julio cesar sanchez" <jc...@gmail.com> wrote:
> I was the other way around, I got Appium working easily, but I was never
> able to make calabash work in iOS because it always failed to instrument my
> app.
> Both worked fine for Android though.
>
>
> 2016-12-07 12:48 GMT+01:00 Trevor Brindle <ta...@gmail.com>:
>
> > As little market share as windows has, it's hard to say how much effort
> > should be dedicated to additional testing for it. That said, if windows
> > testing is deemed necessary, I don't think we have much choice.
> >
> > Just keep in mind the new windows support and new iOS driver are all
> appium
> > 1.6, only released less than 60 days ago.
> >
> > Perhaps a PoC project should be done for picking the appropriate
> > technology?
> >
> > We are midway through a similar PoC at my place of work, I could
> volunteer
> > some time.
> >
> > On Tue, Dec 6, 2016 at 10:02 PM Filip Maj <ma...@gmail.com> wrote:
> >
> > > Full disclosure: I contribute(d) to appium and worked for Sauce Labs,
> > >
> > > so I am pretty biased ;)
> > >
> > >
> > >
> > > Calabash, unfortunately, is iOS and Android only and does not support
> > >
> > > Windows app, whereas Appium does via WinAppDriver [1].
> > >
> > >
> > >
> > > I agree that appium, at least earlier on, was difficult to set up.
> > >
> > > These days it's a simple `npm install`. I think appium has a much more
> > >
> > > prolific committership to boot (MSFT contributes to it, for example).
> > >
> > > The github network stats for each project back that up as well.
> > >
> > >
> > >
> > > [1] https://github.com/microsoft/winappdriver
> > >
> > >
> > >
> > > On Tue, Dec 6, 2016 at 4:02 PM, Trevor Brindle <ta...@gmail.com>
> > > wrote:
> > >
> > > > In my experience with appium, it is troublesome to set up and
> maintain.
> > > We
> > >
> > > > may investigate other frameworks that have similar functionality. I
> > have
> > >
> > > > had great luck with calabash.
> > >
> > > >
> > >
> > > >
> > >
> > > > On Tue, Dec 6, 2016 at 5:30 PM julio cesar sanchez <
> > > jcesarmobile@gmail.com>
> > >
> > > > wrote:
> > >
> > > >
> > >
> > > >> +1 to appium
> > >
> > > >>
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> 2016-12-05 23:33 GMT+01:00 Filip Maj <ma...@gmail.com>:
> > >
> > > >>
> > >
> > > >>
> > >
> > > >>
> > >
> > > >> > Hi, it's me again!
> > >
> > > >>
> > >
> > > >> >
> > >
> > > >>
> > >
> > > >> > How I'd like to contribute to Cordova is to help automate the
> stuff
> > >
> > > >>
> > >
> > > >> > that saves committers having to take the manual time to do
> > themselves.
> > >
> > > >>
> > >
> > > >> > I think a good first goal would be to help automate as much of
> > >
> > > >>
> > >
> > > >> > platform release testing as possible [1]. The autotests seem to be
> > >
> > > >>
> > >
> > > >> > handled relatively well by the CI system so far (if anyone feels
> > this
> > >
> > > >>
> > >
> > > >> > is not true, please speak up! I want to hear about what works and
> > what
> > >
> > > >>
> > >
> > > >> > does not). The other part of mobile-spec-based platform release
> > >
> > > >>
> > >
> > > >> > testing is to "run the manual tests". After reading the
> > >
> > > >>
> > >
> > > >> > cordova-plugin-test-framework README [2], I thought this would be
> a
> > >
> > > >>
> > >
> > > >> > good place to give plugin developers some better tools to deal
> with
> > >
> > > >>
> > >
> > > >> > testing complex UI interactions that the autotests can't handle on
> > >
> > > >>
> > >
> > > >> > their own. I was thinking appium [3] would be a good tool to
> > >
> > > >>
> > >
> > > >> > complement that in this case. It gives us UI hooks into both web
> and
> > >
> > > >>
> > >
> > > >> > natives contexts within hybrid applications, plus it also allows
> us
> > to
> > >
> > > >>
> > >
> > > >> > inject JavaScript into the web context. Wondering what others
> think?
> > >
> > > >>
> > >
> > > >> >
> > >
> > > >>
> > >
> > > >> > I could then foresee, with a little bit of scaffolding, a way to
> > >
> > > >>
> > >
> > > >> > string plugins' appium tests together to fully automate the
> 'manual'
> > >
> > > >>
> > >
> > > >> > testing of plugin tests during platform release testing.
> > >
> > > >>
> > >
> > > >> >
> > >
> > > >>
> > >
> > > >> > Let me know what y'all think!
> > >
> > > >>
> > >
> > > >> >
> > >
> > > >>
> > >
> > > >> > [1] https://github.com/apache/cordova-coho/blob/master/docs/
> > >
> > > >>
> > >
> > > >> > platforms-release-process.md#what-to-test
> > >
> > > >>
> > >
> > > >> > [2] https://github.com/apache/cordova-plugin-test-framework#
> > >
> > > >>
> > >
> > > >> > defining-manual-tests
> > >
> > > >>
> > >
> > > >> > [3] appium.io
> > >
> > > >>
> > >
> > > >> >
> > >
> > > >>
> > >
> > > >> > ------------------------------------------------------------
> > ---------
> > >
> > > >>
> > >
> > > >> > To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
> > >
> > > >>
> > >
> > > >> > For additional commands, e-mail: dev-help@cordova.apache.org
> > >
> > > >>
> > >
> > > >> >
> > >
> > > >>
> > >
> > > >> >
> > >
> > > >>
> > >
> > > >>
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > >
> > > To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
> > >
> > > For additional commands, e-mail: dev-help@cordova.apache.org
> > >
> > >
> > >
> > >
> >
>
Re: An idea for manually testing plugins
Posted by julio cesar sanchez <jc...@gmail.com>.
I was the other way around, I got Appium working easily, but I was never
able to make calabash work in iOS because it always failed to instrument my
app.
Both worked fine for Android though.
2016-12-07 12:48 GMT+01:00 Trevor Brindle <ta...@gmail.com>:
> As little market share as windows has, it's hard to say how much effort
> should be dedicated to additional testing for it. That said, if windows
> testing is deemed necessary, I don't think we have much choice.
>
> Just keep in mind the new windows support and new iOS driver are all appium
> 1.6, only released less than 60 days ago.
>
> Perhaps a PoC project should be done for picking the appropriate
> technology?
>
> We are midway through a similar PoC at my place of work, I could volunteer
> some time.
>
> On Tue, Dec 6, 2016 at 10:02 PM Filip Maj <ma...@gmail.com> wrote:
>
> > Full disclosure: I contribute(d) to appium and worked for Sauce Labs,
> >
> > so I am pretty biased ;)
> >
> >
> >
> > Calabash, unfortunately, is iOS and Android only and does not support
> >
> > Windows app, whereas Appium does via WinAppDriver [1].
> >
> >
> >
> > I agree that appium, at least earlier on, was difficult to set up.
> >
> > These days it's a simple `npm install`. I think appium has a much more
> >
> > prolific committership to boot (MSFT contributes to it, for example).
> >
> > The github network stats for each project back that up as well.
> >
> >
> >
> > [1] https://github.com/microsoft/winappdriver
> >
> >
> >
> > On Tue, Dec 6, 2016 at 4:02 PM, Trevor Brindle <ta...@gmail.com>
> > wrote:
> >
> > > In my experience with appium, it is troublesome to set up and maintain.
> > We
> >
> > > may investigate other frameworks that have similar functionality. I
> have
> >
> > > had great luck with calabash.
> >
> > >
> >
> > >
> >
> > > On Tue, Dec 6, 2016 at 5:30 PM julio cesar sanchez <
> > jcesarmobile@gmail.com>
> >
> > > wrote:
> >
> > >
> >
> > >> +1 to appium
> >
> > >>
> >
> > >>
> >
> > >>
> >
> > >> 2016-12-05 23:33 GMT+01:00 Filip Maj <ma...@gmail.com>:
> >
> > >>
> >
> > >>
> >
> > >>
> >
> > >> > Hi, it's me again!
> >
> > >>
> >
> > >> >
> >
> > >>
> >
> > >> > How I'd like to contribute to Cordova is to help automate the stuff
> >
> > >>
> >
> > >> > that saves committers having to take the manual time to do
> themselves.
> >
> > >>
> >
> > >> > I think a good first goal would be to help automate as much of
> >
> > >>
> >
> > >> > platform release testing as possible [1]. The autotests seem to be
> >
> > >>
> >
> > >> > handled relatively well by the CI system so far (if anyone feels
> this
> >
> > >>
> >
> > >> > is not true, please speak up! I want to hear about what works and
> what
> >
> > >>
> >
> > >> > does not). The other part of mobile-spec-based platform release
> >
> > >>
> >
> > >> > testing is to "run the manual tests". After reading the
> >
> > >>
> >
> > >> > cordova-plugin-test-framework README [2], I thought this would be a
> >
> > >>
> >
> > >> > good place to give plugin developers some better tools to deal with
> >
> > >>
> >
> > >> > testing complex UI interactions that the autotests can't handle on
> >
> > >>
> >
> > >> > their own. I was thinking appium [3] would be a good tool to
> >
> > >>
> >
> > >> > complement that in this case. It gives us UI hooks into both web and
> >
> > >>
> >
> > >> > natives contexts within hybrid applications, plus it also allows us
> to
> >
> > >>
> >
> > >> > inject JavaScript into the web context. Wondering what others think?
> >
> > >>
> >
> > >> >
> >
> > >>
> >
> > >> > I could then foresee, with a little bit of scaffolding, a way to
> >
> > >>
> >
> > >> > string plugins' appium tests together to fully automate the 'manual'
> >
> > >>
> >
> > >> > testing of plugin tests during platform release testing.
> >
> > >>
> >
> > >> >
> >
> > >>
> >
> > >> > Let me know what y'all think!
> >
> > >>
> >
> > >> >
> >
> > >>
> >
> > >> > [1] https://github.com/apache/cordova-coho/blob/master/docs/
> >
> > >>
> >
> > >> > platforms-release-process.md#what-to-test
> >
> > >>
> >
> > >> > [2] https://github.com/apache/cordova-plugin-test-framework#
> >
> > >>
> >
> > >> > defining-manual-tests
> >
> > >>
> >
> > >> > [3] appium.io
> >
> > >>
> >
> > >> >
> >
> > >>
> >
> > >> > ------------------------------------------------------------
> ---------
> >
> > >>
> >
> > >> > To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
> >
> > >>
> >
> > >> > For additional commands, e-mail: dev-help@cordova.apache.org
> >
> > >>
> >
> > >> >
> >
> > >>
> >
> > >> >
> >
> > >>
> >
> > >>
> >
> >
> >
> > ---------------------------------------------------------------------
> >
> > To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
> >
> > For additional commands, e-mail: dev-help@cordova.apache.org
> >
> >
> >
> >
>
Re: An idea for manually testing plugins
Posted by Trevor Brindle <ta...@gmail.com>.
As little market share as windows has, it's hard to say how much effort
should be dedicated to additional testing for it. That said, if windows
testing is deemed necessary, I don't think we have much choice.
Just keep in mind the new windows support and new iOS driver are all appium
1.6, only released less than 60 days ago.
Perhaps a PoC project should be done for picking the appropriate
technology?
We are midway through a similar PoC at my place of work, I could volunteer
some time.
On Tue, Dec 6, 2016 at 10:02 PM Filip Maj <ma...@gmail.com> wrote:
> Full disclosure: I contribute(d) to appium and worked for Sauce Labs,
>
> so I am pretty biased ;)
>
>
>
> Calabash, unfortunately, is iOS and Android only and does not support
>
> Windows app, whereas Appium does via WinAppDriver [1].
>
>
>
> I agree that appium, at least earlier on, was difficult to set up.
>
> These days it's a simple `npm install`. I think appium has a much more
>
> prolific committership to boot (MSFT contributes to it, for example).
>
> The github network stats for each project back that up as well.
>
>
>
> [1] https://github.com/microsoft/winappdriver
>
>
>
> On Tue, Dec 6, 2016 at 4:02 PM, Trevor Brindle <ta...@gmail.com>
> wrote:
>
> > In my experience with appium, it is troublesome to set up and maintain.
> We
>
> > may investigate other frameworks that have similar functionality. I have
>
> > had great luck with calabash.
>
> >
>
> >
>
> > On Tue, Dec 6, 2016 at 5:30 PM julio cesar sanchez <
> jcesarmobile@gmail.com>
>
> > wrote:
>
> >
>
> >> +1 to appium
>
> >>
>
> >>
>
> >>
>
> >> 2016-12-05 23:33 GMT+01:00 Filip Maj <ma...@gmail.com>:
>
> >>
>
> >>
>
> >>
>
> >> > Hi, it's me again!
>
> >>
>
> >> >
>
> >>
>
> >> > How I'd like to contribute to Cordova is to help automate the stuff
>
> >>
>
> >> > that saves committers having to take the manual time to do themselves.
>
> >>
>
> >> > I think a good first goal would be to help automate as much of
>
> >>
>
> >> > platform release testing as possible [1]. The autotests seem to be
>
> >>
>
> >> > handled relatively well by the CI system so far (if anyone feels this
>
> >>
>
> >> > is not true, please speak up! I want to hear about what works and what
>
> >>
>
> >> > does not). The other part of mobile-spec-based platform release
>
> >>
>
> >> > testing is to "run the manual tests". After reading the
>
> >>
>
> >> > cordova-plugin-test-framework README [2], I thought this would be a
>
> >>
>
> >> > good place to give plugin developers some better tools to deal with
>
> >>
>
> >> > testing complex UI interactions that the autotests can't handle on
>
> >>
>
> >> > their own. I was thinking appium [3] would be a good tool to
>
> >>
>
> >> > complement that in this case. It gives us UI hooks into both web and
>
> >>
>
> >> > natives contexts within hybrid applications, plus it also allows us to
>
> >>
>
> >> > inject JavaScript into the web context. Wondering what others think?
>
> >>
>
> >> >
>
> >>
>
> >> > I could then foresee, with a little bit of scaffolding, a way to
>
> >>
>
> >> > string plugins' appium tests together to fully automate the 'manual'
>
> >>
>
> >> > testing of plugin tests during platform release testing.
>
> >>
>
> >> >
>
> >>
>
> >> > Let me know what y'all think!
>
> >>
>
> >> >
>
> >>
>
> >> > [1] https://github.com/apache/cordova-coho/blob/master/docs/
>
> >>
>
> >> > platforms-release-process.md#what-to-test
>
> >>
>
> >> > [2] https://github.com/apache/cordova-plugin-test-framework#
>
> >>
>
> >> > defining-manual-tests
>
> >>
>
> >> > [3] appium.io
>
> >>
>
> >> >
>
> >>
>
> >> > ---------------------------------------------------------------------
>
> >>
>
> >> > To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
>
> >>
>
> >> > For additional commands, e-mail: dev-help@cordova.apache.org
>
> >>
>
> >> >
>
> >>
>
> >> >
>
> >>
>
> >>
>
>
>
> ---------------------------------------------------------------------
>
> To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
>
> For additional commands, e-mail: dev-help@cordova.apache.org
>
>
>
>
Re: An idea for manually testing plugins
Posted by Filip Maj <ma...@gmail.com>.
Full disclosure: I contribute(d) to appium and worked for Sauce Labs,
so I am pretty biased ;)
Calabash, unfortunately, is iOS and Android only and does not support
Windows app, whereas Appium does via WinAppDriver [1].
I agree that appium, at least earlier on, was difficult to set up.
These days it's a simple `npm install`. I think appium has a much more
prolific committership to boot (MSFT contributes to it, for example).
The github network stats for each project back that up as well.
[1] https://github.com/microsoft/winappdriver
On Tue, Dec 6, 2016 at 4:02 PM, Trevor Brindle <ta...@gmail.com> wrote:
> In my experience with appium, it is troublesome to set up and maintain. We
> may investigate other frameworks that have similar functionality. I have
> had great luck with calabash.
>
>
> On Tue, Dec 6, 2016 at 5:30 PM julio cesar sanchez <jc...@gmail.com>
> wrote:
>
>> +1 to appium
>>
>>
>>
>> 2016-12-05 23:33 GMT+01:00 Filip Maj <ma...@gmail.com>:
>>
>>
>>
>> > Hi, it's me again!
>>
>> >
>>
>> > How I'd like to contribute to Cordova is to help automate the stuff
>>
>> > that saves committers having to take the manual time to do themselves.
>>
>> > I think a good first goal would be to help automate as much of
>>
>> > platform release testing as possible [1]. The autotests seem to be
>>
>> > handled relatively well by the CI system so far (if anyone feels this
>>
>> > is not true, please speak up! I want to hear about what works and what
>>
>> > does not). The other part of mobile-spec-based platform release
>>
>> > testing is to "run the manual tests". After reading the
>>
>> > cordova-plugin-test-framework README [2], I thought this would be a
>>
>> > good place to give plugin developers some better tools to deal with
>>
>> > testing complex UI interactions that the autotests can't handle on
>>
>> > their own. I was thinking appium [3] would be a good tool to
>>
>> > complement that in this case. It gives us UI hooks into both web and
>>
>> > natives contexts within hybrid applications, plus it also allows us to
>>
>> > inject JavaScript into the web context. Wondering what others think?
>>
>> >
>>
>> > I could then foresee, with a little bit of scaffolding, a way to
>>
>> > string plugins' appium tests together to fully automate the 'manual'
>>
>> > testing of plugin tests during platform release testing.
>>
>> >
>>
>> > Let me know what y'all think!
>>
>> >
>>
>> > [1] https://github.com/apache/cordova-coho/blob/master/docs/
>>
>> > platforms-release-process.md#what-to-test
>>
>> > [2] https://github.com/apache/cordova-plugin-test-framework#
>>
>> > defining-manual-tests
>>
>> > [3] appium.io
>>
>> >
>>
>> > ---------------------------------------------------------------------
>>
>> > To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
>>
>> > For additional commands, e-mail: dev-help@cordova.apache.org
>>
>> >
>>
>> >
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
For additional commands, e-mail: dev-help@cordova.apache.org
Re: An idea for manually testing plugins
Posted by Trevor Brindle <ta...@gmail.com>.
In my experience with appium, it is troublesome to set up and maintain. We
may investigate other frameworks that have similar functionality. I have
had great luck with calabash.
On Tue, Dec 6, 2016 at 5:30 PM julio cesar sanchez <jc...@gmail.com>
wrote:
> +1 to appium
>
>
>
> 2016-12-05 23:33 GMT+01:00 Filip Maj <ma...@gmail.com>:
>
>
>
> > Hi, it's me again!
>
> >
>
> > How I'd like to contribute to Cordova is to help automate the stuff
>
> > that saves committers having to take the manual time to do themselves.
>
> > I think a good first goal would be to help automate as much of
>
> > platform release testing as possible [1]. The autotests seem to be
>
> > handled relatively well by the CI system so far (if anyone feels this
>
> > is not true, please speak up! I want to hear about what works and what
>
> > does not). The other part of mobile-spec-based platform release
>
> > testing is to "run the manual tests". After reading the
>
> > cordova-plugin-test-framework README [2], I thought this would be a
>
> > good place to give plugin developers some better tools to deal with
>
> > testing complex UI interactions that the autotests can't handle on
>
> > their own. I was thinking appium [3] would be a good tool to
>
> > complement that in this case. It gives us UI hooks into both web and
>
> > natives contexts within hybrid applications, plus it also allows us to
>
> > inject JavaScript into the web context. Wondering what others think?
>
> >
>
> > I could then foresee, with a little bit of scaffolding, a way to
>
> > string plugins' appium tests together to fully automate the 'manual'
>
> > testing of plugin tests during platform release testing.
>
> >
>
> > Let me know what y'all think!
>
> >
>
> > [1] https://github.com/apache/cordova-coho/blob/master/docs/
>
> > platforms-release-process.md#what-to-test
>
> > [2] https://github.com/apache/cordova-plugin-test-framework#
>
> > defining-manual-tests
>
> > [3] appium.io
>
> >
>
> > ---------------------------------------------------------------------
>
> > To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
>
> > For additional commands, e-mail: dev-help@cordova.apache.org
>
> >
>
> >
>
>
Re: An idea for manually testing plugins
Posted by julio cesar sanchez <jc...@gmail.com>.
+1 to appium
2016-12-05 23:33 GMT+01:00 Filip Maj <ma...@gmail.com>:
> Hi, it's me again!
>
> How I'd like to contribute to Cordova is to help automate the stuff
> that saves committers having to take the manual time to do themselves.
> I think a good first goal would be to help automate as much of
> platform release testing as possible [1]. The autotests seem to be
> handled relatively well by the CI system so far (if anyone feels this
> is not true, please speak up! I want to hear about what works and what
> does not). The other part of mobile-spec-based platform release
> testing is to "run the manual tests". After reading the
> cordova-plugin-test-framework README [2], I thought this would be a
> good place to give plugin developers some better tools to deal with
> testing complex UI interactions that the autotests can't handle on
> their own. I was thinking appium [3] would be a good tool to
> complement that in this case. It gives us UI hooks into both web and
> natives contexts within hybrid applications, plus it also allows us to
> inject JavaScript into the web context. Wondering what others think?
>
> I could then foresee, with a little bit of scaffolding, a way to
> string plugins' appium tests together to fully automate the 'manual'
> testing of plugin tests during platform release testing.
>
> Let me know what y'all think!
>
> [1] https://github.com/apache/cordova-coho/blob/master/docs/
> platforms-release-process.md#what-to-test
> [2] https://github.com/apache/cordova-plugin-test-framework#
> defining-manual-tests
> [3] appium.io
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
> For additional commands, e-mail: dev-help@cordova.apache.org
>
>