You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cordova.apache.org by "Alexander Sorokin (Akvelon)" <v-...@microsoft.com> on 2016/01/20 18:11:54 UTC
[DISCUSS] Improving core plugins automated test coverage
Hi guys.
Lately I've been working on improving automated test coverage of core plugins. Work is still in progress but there are some results that I want to share.
The idea is to use Appium framework to automate core plugins manual tests in mobilespec app. I've implemented a new command for Medic. It starts an Appium server instance, looks for the tests, then runs the mobilespec app from the specified folder and runs the tests.
Appium supports only iOS and Android platforms so there are tests only for these two, but we can extend the coverage of other platforms using different ui testing frameworks.
The tests themselves are written for jasmine-node and are using "wd" module to connect to Appium server. They are currently located in plugin repos. Here's some links:
Test runner (medic):
https://github.com/apache/cordova-medic/pull/70
Battery Status tests:
https://github.com/apache/cordova-plugin-battery-status/pull/29
Camera tests:
https://github.com/apache/cordova-plugin-camera/pull/156
Contacts tests:
https://github.com/apache/cordova-plugin-contacts/pull/101
Device Motion tests:
https://github.com/apache/cordova-plugin-device-motion/pull/39
Tests can run on both real devices and emulators/simulators.
While developing the tests, I've been using Android emulator and device with API level 19 (Android 4.4, KitKat) and iOS device (iPad 2) with iOS 7.1 so the tests can fail on other devices/emulators. I'm going to add the support for other versions/devices as well as implement more Appium tests for other plugins in the nearest future.
To run the tests:
1. Clone cordova-medic repo
2. Clone tested plugins repos (appium tests are there, in appium-tests folder)
3. Create mobile spec app with tested plugins:
node cordova-mobile-spec/createmobilespec/createmobilespec.js mobspec-android --android --plugins "cordova-plugin-battery-status cordova-plugin-camera cordova-plugin-contacts cordova-plugin-device-motion"
4. Build the mobilespec app:
cd mobspec-android && cordova build
5. Start the emulator/connect the device
6. Run the tests using command:
node cordova-medic/medic/medic.js appium --app [mobilespec folder] --platform [platform] --device-name [device/emulator name] --platform-version [platform version] --plugins [list of plugins to test separated by space] [--device (only if running on a real device)] --udid [iOS real devices only, device UDID]
Every argument except --platform is optional. On iOS, --udid is mandatory if running on real device.
For example, android emulator:
node cordova-medic/medic/medic.js appium --app mobspec-android --platform android --device-name appium --platform-version 19 --plugins "cordova-plugin-device-motion"
Android device:
node cordova-medic/medic/medic.js appium --platform android --app mobspec-android --plugins "cordova-plugin-contacts" --device
iOS simulator:
node cordova-medic/medic/medic.js appium --app mobspec-ios --platform ios --plugins "cordova-plugin-contacts"
iOS device:
node cordova-medic/medic/medic.js appium --app mobspec-ios --platform ios --device-name "iPad 2" --udid [UDID here] --plugins "cordova-plugin-camera cordova-plugin-contacts"
I hope we will be running these tests along with other medic tests that are now running on CI in future:
http://ci.cordova.io/
Please take a look and say what you think.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
For additional commands, e-mail: dev-help@cordova.apache.org
Re: [DISCUSS] Improving core plugins automated test coverage
Posted by Filip Maj <ma...@gmail.com>.
O hey!
This is pretty sweet, Alexander. It's been so long since I've looked
at medic code, oh my. I feel so old all of a sudden :D
A few notes:
- The way you use appium in here is mostly for navigating the UI -
that is awesome. Totally agree with this approach - automate all the
things so y'all can get closer and closer to a continuous integration
nirvana.
- If you can get an appium run to work locally for you, moving to
Sauce Labs is a breeze - you just change the endpoint of the appium
server from wherever you are running it to ondemand.saucelabs.com:80.
Sauce provides "Open Sauce" for open source projects [1] - which
Cordova certainly falls under. That'll give y'all 5 concurrent VMs to
run tests on. Only iOS and Android are supported right now, but I
think it'd be a big win for Cordova to move to this. You don't have to
worry about managing the emulators/simulators, and parallelizing some
of these functional tests helps speed them up, too, giving you faster
feedback. I'm happy to help getting y'all set up here, too.
- appium, at this time, doesn't expose a command directly to change
the battery level. However, we've filed an issue [2] and hopefully
that will bubble up to the main project in a release or two. So,
functional tests for the battery functionality are probably some of
the more out-of-reach-ones. That said, if you want to use Sauce, and
if we can write the test such that we isolate _just_ the battery
functionality, we can leverage pre-run executables [3] to basically
have an open shell prompt to the VM running the emulator - and `adb`
within it. So we can:
- take those `adb shell set battery blah` and stuff 'em into a
shell script we can host somewhere
- specify the URL to the script as a pre-run executable as part of
the appium test's desired capabilities
- kick off the appium test, which spins up a VM on Sauce's side
- the VM starts the emulator, then runs your pre-run exec scripts,
full of adb commands to tweak the emulator
- finally, the appium test automation commands interacting with
the emulator and the cordova app (presumably with mobile-spec in it?)
start executing, and you get test results back
I think we should start with the simplest APIs possible that are easy
to automate. Build out the framework to do that UI testing - and i
highly recommend we leverage the 5 free VMs from sauce to offload that
- and start integrating that into your day-to-day development work.
Once we've got the basics covered, I'm happy to help work through the
more complex testing cases involving disparate device APIs and see
what we can do on the appium (and/or Sauce) side to help make a
fully-automated testing suite a reality for the cordova project.
Let's do this!
-fil
[1] https://saucelabs.com/opensauce/
[2] https://github.com/appium/appium-adb/issues/133
[3] https://wiki.saucelabs.com/display/DOCS/Test+Configuration+Options#TestConfigurationOptions-Pre-runExecutables
On Fri, Jan 29, 2016 at 4:53 AM, Carlos Santana <cs...@gmail.com> wrote:
> Thanks for the effort into looking into it.
> For now continue as plan, maybe in the future a portion of the tests can be
> ran on Soucelabs if it make sense to complement on what we have with our
> devices and emulators.
> On Fri, Jan 29, 2016 at 3:20 AM Alexander Sorokin (Akvelon) <
> v-alsoro@microsoft.com> wrote:
>
>> It looks like the Saucelabs infrastructure won't suit our needs. A lot of
>> Appium tests for Android platform now need to connect to the emulator
>> directly via adb or telnet to send mock commands.
>>
>> For example, to mock charger in the battery-status tests, we execute "adb
>> shell dumpsys battery set ac 0".
>>
>> It looks like saucelabs won't let us do that, so we're bound to our own
>> devices and emulators.
>>
>> -----Original Message-----
>> From: Alexander Sorokin (Akvelon) [mailto:v-alsoro@microsoft.com]
>> Sent: Friday, January 22, 2016 12:11 PM
>> To: dev@cordova.apache.org
>> Subject: RE: [DISCUSS] Improving core plugins automated test coverage
>>
>> Thanks for suggesting saucelabs! I'll take a look at it.
>>
>> -----Original Message-----
>> From: Carlos Santana [mailto:csantana23@gmail.com]
>> Sent: Thursday, January 21, 2016 4:25 AM
>> To: dev@cordova.apache.org
>> Subject: Re: [DISCUSS] Improving core plugins automated test coverage
>>
>> Wow this is great! not just for our cordova code, but app developers that
>> want to learn how to use appium to test their own plugins and app code
>>
>> Is this something we can run on saucelabs?, I think is free for open
>> source projects.
>>
>>
>> On Wed, Jan 20, 2016 at 8:16 PM Steven Gill <st...@gmail.com>
>> wrote:
>>
>> > Sweet! Thanks for doing this Alexander! Always thought it would be a
>> > great idea to write some our tests with appium
>> >
>> > On Wed, Jan 20, 2016 at 5:07 PM, Joe Bowser <bo...@gmail.com> wrote:
>> >
>> > > So, Appium works now and I can close the issue complaining that
>> > > Appium doesn't work?
>> > >
>> > > On Wed, Jan 20, 2016, 9:12 AM Alexander Sorokin (Akvelon) <
>> > > v-alsoro@microsoft.com> wrote:
>> > >
>> > > > Hi guys.
>> > > >
>> > > > Lately I've been working on improving automated test coverage of
>> > > > core plugins. Work is still in progress but there are some results
>> > > > that I
>> > want
>> > > > to share.
>> > > >
>> > > > The idea is to use Appium framework to automate core plugins
>> > > > manual
>> > tests
>> > > > in mobilespec app. I've implemented a new command for Medic. It
>> > > > starts
>> > an
>> > > > Appium server instance, looks for the tests, then runs the
>> > > > mobilespec
>> > app
>> > > > from the specified folder and runs the tests.
>> > > >
>> > > > Appium supports only iOS and Android platforms so there are tests
>> > > > only
>> > > for
>> > > > these two, but we can extend the coverage of other platforms using
>> > > > different ui testing frameworks.
>> > > >
>> > > > The tests themselves are written for jasmine-node and are using "wd"
>> > > > module to connect to Appium server. They are currently located in
>> > plugin
>> > > > repos. Here's some links:
>> > > >
>> > > > Test runner (medic):
>> > > > https://github.com/apache/cordova-medic/pull/70
>> > > >
>> > > > Battery Status tests:
>> > > > https://github.com/apache/cordova-plugin-battery-status/pull/29
>> > > >
>> > > > Camera tests:
>> > > > https://github.com/apache/cordova-plugin-camera/pull/156
>> > > >
>> > > > Contacts tests:
>> > > > https://github.com/apache/cordova-plugin-contacts/pull/101
>> > > >
>> > > > Device Motion tests:
>> > > > https://github.com/apache/cordova-plugin-device-motion/pull/39
>> > > >
>> > > > Tests can run on both real devices and emulators/simulators.
>> > > > While developing the tests, I've been using Android emulator and
>> > > > device with API level 19 (Android 4.4, KitKat) and iOS device
>> > > > (iPad 2) with
>> > iOS
>> > > > 7.1 so the tests can fail on other devices/emulators. I'm going to
>> > > > add
>> > > the
>> > > > support for other versions/devices as well as implement more
>> > > > Appium
>> > tests
>> > > > for other plugins in the nearest future.
>> > > >
>> > > > To run the tests:
>> > > > 1. Clone cordova-medic repo
>> > > > 2. Clone tested plugins repos (appium tests are there, in
>> > > > appium-tests
>> > > > folder)
>> > > > 3. Create mobile spec app with tested plugins:
>> > > > node
>> > > > cordova-mobile-spec/createmobilespec/createmobilespec.js
>> > > > mobspec-android --android --plugins "cordova-plugin-battery-status
>> > > > cordova-plugin-camera cordova-plugin-contacts
>> > > cordova-plugin-device-motion"
>> > > > 4. Build the mobilespec app:
>> > > > cd mobspec-android && cordova build 5. Start the
>> > > > emulator/connect the device 6. Run the tests using command:
>> > > > node cordova-medic/medic/medic.js appium --app [mobilespec folder]
>> > > > --platform [platform] --device-name [device/emulator name]
>> > > > --platform-version [platform version] --plugins [list of plugins
>> > > > to
>> > test
>> > > > separated by space] [--device (only if running on a real device)]
>> > --udid
>> > > > [iOS real devices only, device UDID] Every argument except
>> > > > --platform is optional. On iOS, --udid is
>> > mandatory
>> > > > if running on real device.
>> > > >
>> > > > For example, android emulator:
>> > > > node cordova-medic/medic/medic.js appium --app mobspec-android
>> > --platform
>> > > > android --device-name appium --platform-version 19 --plugins
>> > > > "cordova-plugin-device-motion"
>> > > >
>> > > > Android device:
>> > > > node cordova-medic/medic/medic.js appium --platform android --app
>> > > > mobspec-android --plugins "cordova-plugin-contacts" --device
>> > > >
>> > > > iOS simulator:
>> > > > node cordova-medic/medic/medic.js appium --app mobspec-ios
>> > > > --platform
>> > ios
>> > > > --plugins "cordova-plugin-contacts"
>> > > >
>> > > > iOS device:
>> > > > node cordova-medic/medic/medic.js appium --app mobspec-ios
>> > > > --platform
>> > ios
>> > > > --device-name "iPad 2" --udid [UDID here] --plugins
>> > > "cordova-plugin-camera
>> > > > cordova-plugin-contacts"
>> > > >
>> > > > I hope we will be running these tests along with other medic tests
>> > > > that are now running on CI in future:
>> > > > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fci
>> > > > .cordova.io%2f&data=01%7c01%7cv-alsoro%40microsoft.com%7c5c198a04b
>> > > > bef4dd0eab508d32201c51c%7c72f988bf86f141af91ab2d7cd011db47%7c1&sda
>> > > > ta=YrNTt%2f1zkrRjqr3WVSDF37SDgkCbD2KjjDnaxn47bLU%3d
>> > > >
>> > > > Please take a look and say what you think.
>> > > >
>> > > > ------------------------------------------------------------------
>> > > > --- To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
>> > > > For additional commands, e-mail: dev-help@cordova.apache.org
>> > > >
>> > > >
>> > >
>> >
>> B KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB
>> [ X ܚX K K[XZ[
>> ] ][ X ܚX P ܙ ݘK \ X K ܙ B ܈ Y ] [ۘ[ [X[ K[XZ[
>> ] Z [ ܙ ݘK \ X K ܙ B
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
For additional commands, e-mail: dev-help@cordova.apache.org
Re: [DISCUSS] Improving core plugins automated test coverage
Posted by Carlos Santana <cs...@gmail.com>.
Thanks for the effort into looking into it.
For now continue as plan, maybe in the future a portion of the tests can be
ran on Soucelabs if it make sense to complement on what we have with our
devices and emulators.
On Fri, Jan 29, 2016 at 3:20 AM Alexander Sorokin (Akvelon) <
v-alsoro@microsoft.com> wrote:
> It looks like the Saucelabs infrastructure won't suit our needs. A lot of
> Appium tests for Android platform now need to connect to the emulator
> directly via adb or telnet to send mock commands.
>
> For example, to mock charger in the battery-status tests, we execute "adb
> shell dumpsys battery set ac 0".
>
> It looks like saucelabs won't let us do that, so we're bound to our own
> devices and emulators.
>
> -----Original Message-----
> From: Alexander Sorokin (Akvelon) [mailto:v-alsoro@microsoft.com]
> Sent: Friday, January 22, 2016 12:11 PM
> To: dev@cordova.apache.org
> Subject: RE: [DISCUSS] Improving core plugins automated test coverage
>
> Thanks for suggesting saucelabs! I'll take a look at it.
>
> -----Original Message-----
> From: Carlos Santana [mailto:csantana23@gmail.com]
> Sent: Thursday, January 21, 2016 4:25 AM
> To: dev@cordova.apache.org
> Subject: Re: [DISCUSS] Improving core plugins automated test coverage
>
> Wow this is great! not just for our cordova code, but app developers that
> want to learn how to use appium to test their own plugins and app code
>
> Is this something we can run on saucelabs?, I think is free for open
> source projects.
>
>
> On Wed, Jan 20, 2016 at 8:16 PM Steven Gill <st...@gmail.com>
> wrote:
>
> > Sweet! Thanks for doing this Alexander! Always thought it would be a
> > great idea to write some our tests with appium
> >
> > On Wed, Jan 20, 2016 at 5:07 PM, Joe Bowser <bo...@gmail.com> wrote:
> >
> > > So, Appium works now and I can close the issue complaining that
> > > Appium doesn't work?
> > >
> > > On Wed, Jan 20, 2016, 9:12 AM Alexander Sorokin (Akvelon) <
> > > v-alsoro@microsoft.com> wrote:
> > >
> > > > Hi guys.
> > > >
> > > > Lately I've been working on improving automated test coverage of
> > > > core plugins. Work is still in progress but there are some results
> > > > that I
> > want
> > > > to share.
> > > >
> > > > The idea is to use Appium framework to automate core plugins
> > > > manual
> > tests
> > > > in mobilespec app. I've implemented a new command for Medic. It
> > > > starts
> > an
> > > > Appium server instance, looks for the tests, then runs the
> > > > mobilespec
> > app
> > > > from the specified folder and runs the tests.
> > > >
> > > > Appium supports only iOS and Android platforms so there are tests
> > > > only
> > > for
> > > > these two, but we can extend the coverage of other platforms using
> > > > different ui testing frameworks.
> > > >
> > > > The tests themselves are written for jasmine-node and are using "wd"
> > > > module to connect to Appium server. They are currently located in
> > plugin
> > > > repos. Here's some links:
> > > >
> > > > Test runner (medic):
> > > > https://github.com/apache/cordova-medic/pull/70
> > > >
> > > > Battery Status tests:
> > > > https://github.com/apache/cordova-plugin-battery-status/pull/29
> > > >
> > > > Camera tests:
> > > > https://github.com/apache/cordova-plugin-camera/pull/156
> > > >
> > > > Contacts tests:
> > > > https://github.com/apache/cordova-plugin-contacts/pull/101
> > > >
> > > > Device Motion tests:
> > > > https://github.com/apache/cordova-plugin-device-motion/pull/39
> > > >
> > > > Tests can run on both real devices and emulators/simulators.
> > > > While developing the tests, I've been using Android emulator and
> > > > device with API level 19 (Android 4.4, KitKat) and iOS device
> > > > (iPad 2) with
> > iOS
> > > > 7.1 so the tests can fail on other devices/emulators. I'm going to
> > > > add
> > > the
> > > > support for other versions/devices as well as implement more
> > > > Appium
> > tests
> > > > for other plugins in the nearest future.
> > > >
> > > > To run the tests:
> > > > 1. Clone cordova-medic repo
> > > > 2. Clone tested plugins repos (appium tests are there, in
> > > > appium-tests
> > > > folder)
> > > > 3. Create mobile spec app with tested plugins:
> > > > node
> > > > cordova-mobile-spec/createmobilespec/createmobilespec.js
> > > > mobspec-android --android --plugins "cordova-plugin-battery-status
> > > > cordova-plugin-camera cordova-plugin-contacts
> > > cordova-plugin-device-motion"
> > > > 4. Build the mobilespec app:
> > > > cd mobspec-android && cordova build 5. Start the
> > > > emulator/connect the device 6. Run the tests using command:
> > > > node cordova-medic/medic/medic.js appium --app [mobilespec folder]
> > > > --platform [platform] --device-name [device/emulator name]
> > > > --platform-version [platform version] --plugins [list of plugins
> > > > to
> > test
> > > > separated by space] [--device (only if running on a real device)]
> > --udid
> > > > [iOS real devices only, device UDID] Every argument except
> > > > --platform is optional. On iOS, --udid is
> > mandatory
> > > > if running on real device.
> > > >
> > > > For example, android emulator:
> > > > node cordova-medic/medic/medic.js appium --app mobspec-android
> > --platform
> > > > android --device-name appium --platform-version 19 --plugins
> > > > "cordova-plugin-device-motion"
> > > >
> > > > Android device:
> > > > node cordova-medic/medic/medic.js appium --platform android --app
> > > > mobspec-android --plugins "cordova-plugin-contacts" --device
> > > >
> > > > iOS simulator:
> > > > node cordova-medic/medic/medic.js appium --app mobspec-ios
> > > > --platform
> > ios
> > > > --plugins "cordova-plugin-contacts"
> > > >
> > > > iOS device:
> > > > node cordova-medic/medic/medic.js appium --app mobspec-ios
> > > > --platform
> > ios
> > > > --device-name "iPad 2" --udid [UDID here] --plugins
> > > "cordova-plugin-camera
> > > > cordova-plugin-contacts"
> > > >
> > > > I hope we will be running these tests along with other medic tests
> > > > that are now running on CI in future:
> > > > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fci
> > > > .cordova.io%2f&data=01%7c01%7cv-alsoro%40microsoft.com%7c5c198a04b
> > > > bef4dd0eab508d32201c51c%7c72f988bf86f141af91ab2d7cd011db47%7c1&sda
> > > > ta=YrNTt%2f1zkrRjqr3WVSDF37SDgkCbD2KjjDnaxn47bLU%3d
> > > >
> > > > Please take a look and say what you think.
> > > >
> > > > ------------------------------------------------------------------
> > > > --- To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
> > > > For additional commands, e-mail: dev-help@cordova.apache.org
> > > >
> > > >
> > >
> >
> B KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB
> [ X ܚX K K[XZ[
> ] ][ X ܚX P ܙ ݘK \ X K ܙ B ܈ Y ] [ۘ[ [X[ K[XZ[
> ] Z [ ܙ ݘK \ X K ܙ B
>
RE: [DISCUSS] Improving core plugins automated test coverage
Posted by "Alexander Sorokin (Akvelon)" <v-...@microsoft.com>.
It looks like the Saucelabs infrastructure won't suit our needs. A lot of Appium tests for Android platform now need to connect to the emulator directly via adb or telnet to send mock commands.
For example, to mock charger in the battery-status tests, we execute "adb shell dumpsys battery set ac 0".
It looks like saucelabs won't let us do that, so we're bound to our own devices and emulators.
-----Original Message-----
From: Alexander Sorokin (Akvelon) [mailto:v-alsoro@microsoft.com]
Sent: Friday, January 22, 2016 12:11 PM
To: dev@cordova.apache.org
Subject: RE: [DISCUSS] Improving core plugins automated test coverage
Thanks for suggesting saucelabs! I'll take a look at it.
-----Original Message-----
From: Carlos Santana [mailto:csantana23@gmail.com]
Sent: Thursday, January 21, 2016 4:25 AM
To: dev@cordova.apache.org
Subject: Re: [DISCUSS] Improving core plugins automated test coverage
Wow this is great! not just for our cordova code, but app developers that want to learn how to use appium to test their own plugins and app code
Is this something we can run on saucelabs?, I think is free for open source projects.
On Wed, Jan 20, 2016 at 8:16 PM Steven Gill <st...@gmail.com> wrote:
> Sweet! Thanks for doing this Alexander! Always thought it would be a
> great idea to write some our tests with appium
>
> On Wed, Jan 20, 2016 at 5:07 PM, Joe Bowser <bo...@gmail.com> wrote:
>
> > So, Appium works now and I can close the issue complaining that
> > Appium doesn't work?
> >
> > On Wed, Jan 20, 2016, 9:12 AM Alexander Sorokin (Akvelon) <
> > v-alsoro@microsoft.com> wrote:
> >
> > > Hi guys.
> > >
> > > Lately I've been working on improving automated test coverage of
> > > core plugins. Work is still in progress but there are some results
> > > that I
> want
> > > to share.
> > >
> > > The idea is to use Appium framework to automate core plugins
> > > manual
> tests
> > > in mobilespec app. I've implemented a new command for Medic. It
> > > starts
> an
> > > Appium server instance, looks for the tests, then runs the
> > > mobilespec
> app
> > > from the specified folder and runs the tests.
> > >
> > > Appium supports only iOS and Android platforms so there are tests
> > > only
> > for
> > > these two, but we can extend the coverage of other platforms using
> > > different ui testing frameworks.
> > >
> > > The tests themselves are written for jasmine-node and are using "wd"
> > > module to connect to Appium server. They are currently located in
> plugin
> > > repos. Here's some links:
> > >
> > > Test runner (medic):
> > > https://github.com/apache/cordova-medic/pull/70
> > >
> > > Battery Status tests:
> > > https://github.com/apache/cordova-plugin-battery-status/pull/29
> > >
> > > Camera tests:
> > > https://github.com/apache/cordova-plugin-camera/pull/156
> > >
> > > Contacts tests:
> > > https://github.com/apache/cordova-plugin-contacts/pull/101
> > >
> > > Device Motion tests:
> > > https://github.com/apache/cordova-plugin-device-motion/pull/39
> > >
> > > Tests can run on both real devices and emulators/simulators.
> > > While developing the tests, I've been using Android emulator and
> > > device with API level 19 (Android 4.4, KitKat) and iOS device
> > > (iPad 2) with
> iOS
> > > 7.1 so the tests can fail on other devices/emulators. I'm going to
> > > add
> > the
> > > support for other versions/devices as well as implement more
> > > Appium
> tests
> > > for other plugins in the nearest future.
> > >
> > > To run the tests:
> > > 1. Clone cordova-medic repo
> > > 2. Clone tested plugins repos (appium tests are there, in
> > > appium-tests
> > > folder)
> > > 3. Create mobile spec app with tested plugins:
> > > node
> > > cordova-mobile-spec/createmobilespec/createmobilespec.js
> > > mobspec-android --android --plugins "cordova-plugin-battery-status
> > > cordova-plugin-camera cordova-plugin-contacts
> > cordova-plugin-device-motion"
> > > 4. Build the mobilespec app:
> > > cd mobspec-android && cordova build 5. Start the
> > > emulator/connect the device 6. Run the tests using command:
> > > node cordova-medic/medic/medic.js appium --app [mobilespec folder]
> > > --platform [platform] --device-name [device/emulator name]
> > > --platform-version [platform version] --plugins [list of plugins
> > > to
> test
> > > separated by space] [--device (only if running on a real device)]
> --udid
> > > [iOS real devices only, device UDID] Every argument except
> > > --platform is optional. On iOS, --udid is
> mandatory
> > > if running on real device.
> > >
> > > For example, android emulator:
> > > node cordova-medic/medic/medic.js appium --app mobspec-android
> --platform
> > > android --device-name appium --platform-version 19 --plugins
> > > "cordova-plugin-device-motion"
> > >
> > > Android device:
> > > node cordova-medic/medic/medic.js appium --platform android --app
> > > mobspec-android --plugins "cordova-plugin-contacts" --device
> > >
> > > iOS simulator:
> > > node cordova-medic/medic/medic.js appium --app mobspec-ios
> > > --platform
> ios
> > > --plugins "cordova-plugin-contacts"
> > >
> > > iOS device:
> > > node cordova-medic/medic/medic.js appium --app mobspec-ios
> > > --platform
> ios
> > > --device-name "iPad 2" --udid [UDID here] --plugins
> > "cordova-plugin-camera
> > > cordova-plugin-contacts"
> > >
> > > I hope we will be running these tests along with other medic tests
> > > that are now running on CI in future:
> > > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fci
> > > .cordova.io%2f&data=01%7c01%7cv-alsoro%40microsoft.com%7c5c198a04b
> > > bef4dd0eab508d32201c51c%7c72f988bf86f141af91ab2d7cd011db47%7c1&sda
> > > ta=YrNTt%2f1zkrRjqr3WVSDF37SDgkCbD2KjjDnaxn47bLU%3d
> > >
> > > Please take a look and say what you think.
> > >
> > > ------------------------------------------------------------------
> > > --- To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
> > > For additional commands, e-mail: dev-help@cordova.apache.org
> > >
> > >
> >
>
B KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB [ X ܚX KK[XZ[
] ][ X ܚX P ܙݘK \X K ܙ B ܈Y][ۘ[ [X[ K[XZ[
] Z[ ܙݘK \X K ܙ B
RE: [DISCUSS] Improving core plugins automated test coverage
Posted by "Alexander Sorokin (Akvelon)" <v-...@microsoft.com>.
Thanks for suggesting saucelabs! I'll take a look at it.
-----Original Message-----
From: Carlos Santana [mailto:csantana23@gmail.com]
Sent: Thursday, January 21, 2016 4:25 AM
To: dev@cordova.apache.org
Subject: Re: [DISCUSS] Improving core plugins automated test coverage
Wow this is great! not just for our cordova code, but app developers that want to learn how to use appium to test their own plugins and app code
Is this something we can run on saucelabs?, I think is free for open source projects.
On Wed, Jan 20, 2016 at 8:16 PM Steven Gill <st...@gmail.com> wrote:
> Sweet! Thanks for doing this Alexander! Always thought it would be a
> great idea to write some our tests with appium
>
> On Wed, Jan 20, 2016 at 5:07 PM, Joe Bowser <bo...@gmail.com> wrote:
>
> > So, Appium works now and I can close the issue complaining that
> > Appium doesn't work?
> >
> > On Wed, Jan 20, 2016, 9:12 AM Alexander Sorokin (Akvelon) <
> > v-alsoro@microsoft.com> wrote:
> >
> > > Hi guys.
> > >
> > > Lately I've been working on improving automated test coverage of
> > > core plugins. Work is still in progress but there are some results
> > > that I
> want
> > > to share.
> > >
> > > The idea is to use Appium framework to automate core plugins
> > > manual
> tests
> > > in mobilespec app. I've implemented a new command for Medic. It
> > > starts
> an
> > > Appium server instance, looks for the tests, then runs the
> > > mobilespec
> app
> > > from the specified folder and runs the tests.
> > >
> > > Appium supports only iOS and Android platforms so there are tests
> > > only
> > for
> > > these two, but we can extend the coverage of other platforms using
> > > different ui testing frameworks.
> > >
> > > The tests themselves are written for jasmine-node and are using "wd"
> > > module to connect to Appium server. They are currently located in
> plugin
> > > repos. Here's some links:
> > >
> > > Test runner (medic):
> > > https://github.com/apache/cordova-medic/pull/70
> > >
> > > Battery Status tests:
> > > https://github.com/apache/cordova-plugin-battery-status/pull/29
> > >
> > > Camera tests:
> > > https://github.com/apache/cordova-plugin-camera/pull/156
> > >
> > > Contacts tests:
> > > https://github.com/apache/cordova-plugin-contacts/pull/101
> > >
> > > Device Motion tests:
> > > https://github.com/apache/cordova-plugin-device-motion/pull/39
> > >
> > > Tests can run on both real devices and emulators/simulators.
> > > While developing the tests, I've been using Android emulator and
> > > device with API level 19 (Android 4.4, KitKat) and iOS device
> > > (iPad 2) with
> iOS
> > > 7.1 so the tests can fail on other devices/emulators. I'm going to
> > > add
> > the
> > > support for other versions/devices as well as implement more
> > > Appium
> tests
> > > for other plugins in the nearest future.
> > >
> > > To run the tests:
> > > 1. Clone cordova-medic repo
> > > 2. Clone tested plugins repos (appium tests are there, in
> > > appium-tests
> > > folder)
> > > 3. Create mobile spec app with tested plugins:
> > > node
> > > cordova-mobile-spec/createmobilespec/createmobilespec.js
> > > mobspec-android --android --plugins "cordova-plugin-battery-status
> > > cordova-plugin-camera cordova-plugin-contacts
> > cordova-plugin-device-motion"
> > > 4. Build the mobilespec app:
> > > cd mobspec-android && cordova build 5. Start the
> > > emulator/connect the device 6. Run the tests using command:
> > > node cordova-medic/medic/medic.js appium --app [mobilespec folder]
> > > --platform [platform] --device-name [device/emulator name]
> > > --platform-version [platform version] --plugins [list of plugins
> > > to
> test
> > > separated by space] [--device (only if running on a real device)]
> --udid
> > > [iOS real devices only, device UDID] Every argument except
> > > --platform is optional. On iOS, --udid is
> mandatory
> > > if running on real device.
> > >
> > > For example, android emulator:
> > > node cordova-medic/medic/medic.js appium --app mobspec-android
> --platform
> > > android --device-name appium --platform-version 19 --plugins
> > > "cordova-plugin-device-motion"
> > >
> > > Android device:
> > > node cordova-medic/medic/medic.js appium --platform android --app
> > > mobspec-android --plugins "cordova-plugin-contacts" --device
> > >
> > > iOS simulator:
> > > node cordova-medic/medic/medic.js appium --app mobspec-ios
> > > --platform
> ios
> > > --plugins "cordova-plugin-contacts"
> > >
> > > iOS device:
> > > node cordova-medic/medic/medic.js appium --app mobspec-ios
> > > --platform
> ios
> > > --device-name "iPad 2" --udid [UDID here] --plugins
> > "cordova-plugin-camera
> > > cordova-plugin-contacts"
> > >
> > > I hope we will be running these tests along with other medic tests
> > > that are now running on CI in future:
> > > https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fci
> > > .cordova.io%2f&data=01%7c01%7cv-alsoro%40microsoft.com%7c5c198a04b
> > > bef4dd0eab508d32201c51c%7c72f988bf86f141af91ab2d7cd011db47%7c1&sda
> > > ta=YrNTt%2f1zkrRjqr3WVSDF37SDgkCbD2KjjDnaxn47bLU%3d
> > >
> > > Please take a look and say what you think.
> > >
> > > ------------------------------------------------------------------
> > > --- To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
> > > For additional commands, e-mail: dev-help@cordova.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] Improving core plugins automated test coverage
Posted by Carlos Santana <cs...@gmail.com>.
Wow this is great! not just for our cordova code, but app developers that
want to learn how to use appium to test their own plugins and app code
Is this something we can run on saucelabs?, I think is free for open source
projects.
On Wed, Jan 20, 2016 at 8:16 PM Steven Gill <st...@gmail.com> wrote:
> Sweet! Thanks for doing this Alexander! Always thought it would be a great
> idea to write some our tests with appium
>
> On Wed, Jan 20, 2016 at 5:07 PM, Joe Bowser <bo...@gmail.com> wrote:
>
> > So, Appium works now and I can close the issue complaining that Appium
> > doesn't work?
> >
> > On Wed, Jan 20, 2016, 9:12 AM Alexander Sorokin (Akvelon) <
> > v-alsoro@microsoft.com> wrote:
> >
> > > Hi guys.
> > >
> > > Lately I've been working on improving automated test coverage of core
> > > plugins. Work is still in progress but there are some results that I
> want
> > > to share.
> > >
> > > The idea is to use Appium framework to automate core plugins manual
> tests
> > > in mobilespec app. I've implemented a new command for Medic. It starts
> an
> > > Appium server instance, looks for the tests, then runs the mobilespec
> app
> > > from the specified folder and runs the tests.
> > >
> > > Appium supports only iOS and Android platforms so there are tests only
> > for
> > > these two, but we can extend the coverage of other platforms using
> > > different ui testing frameworks.
> > >
> > > The tests themselves are written for jasmine-node and are using "wd"
> > > module to connect to Appium server. They are currently located in
> plugin
> > > repos. Here's some links:
> > >
> > > Test runner (medic):
> > > https://github.com/apache/cordova-medic/pull/70
> > >
> > > Battery Status tests:
> > > https://github.com/apache/cordova-plugin-battery-status/pull/29
> > >
> > > Camera tests:
> > > https://github.com/apache/cordova-plugin-camera/pull/156
> > >
> > > Contacts tests:
> > > https://github.com/apache/cordova-plugin-contacts/pull/101
> > >
> > > Device Motion tests:
> > > https://github.com/apache/cordova-plugin-device-motion/pull/39
> > >
> > > Tests can run on both real devices and emulators/simulators.
> > > While developing the tests, I've been using Android emulator and device
> > > with API level 19 (Android 4.4, KitKat) and iOS device (iPad 2) with
> iOS
> > > 7.1 so the tests can fail on other devices/emulators. I'm going to add
> > the
> > > support for other versions/devices as well as implement more Appium
> tests
> > > for other plugins in the nearest future.
> > >
> > > To run the tests:
> > > 1. Clone cordova-medic repo
> > > 2. Clone tested plugins repos (appium tests are there, in appium-tests
> > > folder)
> > > 3. Create mobile spec app with tested plugins:
> > > node cordova-mobile-spec/createmobilespec/createmobilespec.js
> > > mobspec-android --android --plugins "cordova-plugin-battery-status
> > > cordova-plugin-camera cordova-plugin-contacts
> > cordova-plugin-device-motion"
> > > 4. Build the mobilespec app:
> > > cd mobspec-android && cordova build
> > > 5. Start the emulator/connect the device
> > > 6. Run the tests using command:
> > > node cordova-medic/medic/medic.js appium --app [mobilespec folder]
> > > --platform [platform] --device-name [device/emulator name]
> > > --platform-version [platform version] --plugins [list of plugins to
> test
> > > separated by space] [--device (only if running on a real device)]
> --udid
> > > [iOS real devices only, device UDID]
> > > Every argument except --platform is optional. On iOS, --udid is
> mandatory
> > > if running on real device.
> > >
> > > For example, android emulator:
> > > node cordova-medic/medic/medic.js appium --app mobspec-android
> --platform
> > > android --device-name appium --platform-version 19 --plugins
> > > "cordova-plugin-device-motion"
> > >
> > > Android device:
> > > node cordova-medic/medic/medic.js appium --platform android --app
> > > mobspec-android --plugins "cordova-plugin-contacts" --device
> > >
> > > iOS simulator:
> > > node cordova-medic/medic/medic.js appium --app mobspec-ios --platform
> ios
> > > --plugins "cordova-plugin-contacts"
> > >
> > > iOS device:
> > > node cordova-medic/medic/medic.js appium --app mobspec-ios --platform
> ios
> > > --device-name "iPad 2" --udid [UDID here] --plugins
> > "cordova-plugin-camera
> > > cordova-plugin-contacts"
> > >
> > > I hope we will be running these tests along with other medic tests that
> > > are now running on CI in future:
> > > http://ci.cordova.io/
> > >
> > > Please take a look and say what you think.
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
> > > For additional commands, e-mail: dev-help@cordova.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] Improving core plugins automated test coverage
Posted by Steven Gill <st...@gmail.com>.
Sweet! Thanks for doing this Alexander! Always thought it would be a great
idea to write some our tests with appium
On Wed, Jan 20, 2016 at 5:07 PM, Joe Bowser <bo...@gmail.com> wrote:
> So, Appium works now and I can close the issue complaining that Appium
> doesn't work?
>
> On Wed, Jan 20, 2016, 9:12 AM Alexander Sorokin (Akvelon) <
> v-alsoro@microsoft.com> wrote:
>
> > Hi guys.
> >
> > Lately I've been working on improving automated test coverage of core
> > plugins. Work is still in progress but there are some results that I want
> > to share.
> >
> > The idea is to use Appium framework to automate core plugins manual tests
> > in mobilespec app. I've implemented a new command for Medic. It starts an
> > Appium server instance, looks for the tests, then runs the mobilespec app
> > from the specified folder and runs the tests.
> >
> > Appium supports only iOS and Android platforms so there are tests only
> for
> > these two, but we can extend the coverage of other platforms using
> > different ui testing frameworks.
> >
> > The tests themselves are written for jasmine-node and are using "wd"
> > module to connect to Appium server. They are currently located in plugin
> > repos. Here's some links:
> >
> > Test runner (medic):
> > https://github.com/apache/cordova-medic/pull/70
> >
> > Battery Status tests:
> > https://github.com/apache/cordova-plugin-battery-status/pull/29
> >
> > Camera tests:
> > https://github.com/apache/cordova-plugin-camera/pull/156
> >
> > Contacts tests:
> > https://github.com/apache/cordova-plugin-contacts/pull/101
> >
> > Device Motion tests:
> > https://github.com/apache/cordova-plugin-device-motion/pull/39
> >
> > Tests can run on both real devices and emulators/simulators.
> > While developing the tests, I've been using Android emulator and device
> > with API level 19 (Android 4.4, KitKat) and iOS device (iPad 2) with iOS
> > 7.1 so the tests can fail on other devices/emulators. I'm going to add
> the
> > support for other versions/devices as well as implement more Appium tests
> > for other plugins in the nearest future.
> >
> > To run the tests:
> > 1. Clone cordova-medic repo
> > 2. Clone tested plugins repos (appium tests are there, in appium-tests
> > folder)
> > 3. Create mobile spec app with tested plugins:
> > node cordova-mobile-spec/createmobilespec/createmobilespec.js
> > mobspec-android --android --plugins "cordova-plugin-battery-status
> > cordova-plugin-camera cordova-plugin-contacts
> cordova-plugin-device-motion"
> > 4. Build the mobilespec app:
> > cd mobspec-android && cordova build
> > 5. Start the emulator/connect the device
> > 6. Run the tests using command:
> > node cordova-medic/medic/medic.js appium --app [mobilespec folder]
> > --platform [platform] --device-name [device/emulator name]
> > --platform-version [platform version] --plugins [list of plugins to test
> > separated by space] [--device (only if running on a real device)] --udid
> > [iOS real devices only, device UDID]
> > Every argument except --platform is optional. On iOS, --udid is mandatory
> > if running on real device.
> >
> > For example, android emulator:
> > node cordova-medic/medic/medic.js appium --app mobspec-android --platform
> > android --device-name appium --platform-version 19 --plugins
> > "cordova-plugin-device-motion"
> >
> > Android device:
> > node cordova-medic/medic/medic.js appium --platform android --app
> > mobspec-android --plugins "cordova-plugin-contacts" --device
> >
> > iOS simulator:
> > node cordova-medic/medic/medic.js appium --app mobspec-ios --platform ios
> > --plugins "cordova-plugin-contacts"
> >
> > iOS device:
> > node cordova-medic/medic/medic.js appium --app mobspec-ios --platform ios
> > --device-name "iPad 2" --udid [UDID here] --plugins
> "cordova-plugin-camera
> > cordova-plugin-contacts"
> >
> > I hope we will be running these tests along with other medic tests that
> > are now running on CI in future:
> > http://ci.cordova.io/
> >
> > Please take a look and say what you think.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
> > For additional commands, e-mail: dev-help@cordova.apache.org
> >
> >
>
Re: [DISCUSS] Improving core plugins automated test coverage
Posted by Joe Bowser <bo...@gmail.com>.
So, Appium works now and I can close the issue complaining that Appium
doesn't work?
On Wed, Jan 20, 2016, 9:12 AM Alexander Sorokin (Akvelon) <
v-alsoro@microsoft.com> wrote:
> Hi guys.
>
> Lately I've been working on improving automated test coverage of core
> plugins. Work is still in progress but there are some results that I want
> to share.
>
> The idea is to use Appium framework to automate core plugins manual tests
> in mobilespec app. I've implemented a new command for Medic. It starts an
> Appium server instance, looks for the tests, then runs the mobilespec app
> from the specified folder and runs the tests.
>
> Appium supports only iOS and Android platforms so there are tests only for
> these two, but we can extend the coverage of other platforms using
> different ui testing frameworks.
>
> The tests themselves are written for jasmine-node and are using "wd"
> module to connect to Appium server. They are currently located in plugin
> repos. Here's some links:
>
> Test runner (medic):
> https://github.com/apache/cordova-medic/pull/70
>
> Battery Status tests:
> https://github.com/apache/cordova-plugin-battery-status/pull/29
>
> Camera tests:
> https://github.com/apache/cordova-plugin-camera/pull/156
>
> Contacts tests:
> https://github.com/apache/cordova-plugin-contacts/pull/101
>
> Device Motion tests:
> https://github.com/apache/cordova-plugin-device-motion/pull/39
>
> Tests can run on both real devices and emulators/simulators.
> While developing the tests, I've been using Android emulator and device
> with API level 19 (Android 4.4, KitKat) and iOS device (iPad 2) with iOS
> 7.1 so the tests can fail on other devices/emulators. I'm going to add the
> support for other versions/devices as well as implement more Appium tests
> for other plugins in the nearest future.
>
> To run the tests:
> 1. Clone cordova-medic repo
> 2. Clone tested plugins repos (appium tests are there, in appium-tests
> folder)
> 3. Create mobile spec app with tested plugins:
> node cordova-mobile-spec/createmobilespec/createmobilespec.js
> mobspec-android --android --plugins "cordova-plugin-battery-status
> cordova-plugin-camera cordova-plugin-contacts cordova-plugin-device-motion"
> 4. Build the mobilespec app:
> cd mobspec-android && cordova build
> 5. Start the emulator/connect the device
> 6. Run the tests using command:
> node cordova-medic/medic/medic.js appium --app [mobilespec folder]
> --platform [platform] --device-name [device/emulator name]
> --platform-version [platform version] --plugins [list of plugins to test
> separated by space] [--device (only if running on a real device)] --udid
> [iOS real devices only, device UDID]
> Every argument except --platform is optional. On iOS, --udid is mandatory
> if running on real device.
>
> For example, android emulator:
> node cordova-medic/medic/medic.js appium --app mobspec-android --platform
> android --device-name appium --platform-version 19 --plugins
> "cordova-plugin-device-motion"
>
> Android device:
> node cordova-medic/medic/medic.js appium --platform android --app
> mobspec-android --plugins "cordova-plugin-contacts" --device
>
> iOS simulator:
> node cordova-medic/medic/medic.js appium --app mobspec-ios --platform ios
> --plugins "cordova-plugin-contacts"
>
> iOS device:
> node cordova-medic/medic/medic.js appium --app mobspec-ios --platform ios
> --device-name "iPad 2" --udid [UDID here] --plugins "cordova-plugin-camera
> cordova-plugin-contacts"
>
> I hope we will be running these tests along with other medic tests that
> are now running on CI in future:
> http://ci.cordova.io/
>
> Please take a look and say what you think.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
> For additional commands, e-mail: dev-help@cordova.apache.org
>
>