Update on Mobile Spec

[Michal Mocny <mm...@google.com>]

Thanks a bunch to Stacy & Martin (& Jesse) for knocking back a lot of the
work to land new-style plugin tests in the last weeks.

I've done some final cleanup this week, and think that we are for the large
part finally in the new world!  Woo hoo!


A few questions:
- Since many new-style tests landed, and old ones were removed, I went
ahead and closed off the JIRA issues.  If there is outstanding work to be
done, we can reopen.  I assume there isn't, because that would imply that
there are tests that have gone temporarily missing from any flow (bad!).

- I noticed during cleanup that mobile-spec was never running whitelist
tests as part of the auto test suite.  So I fixed the bug.  Except
whitelist tests don't all pass on Android.  Were those test purposely not
being run?

Clearly the current test-harness app is ugly and needs work, and Jason (the
newest member over at Google) has volunteered to pick up work on adding a
few improvements here as he ramps up on cordova stuff.

-Michal

Re: Update on Mobile Spec

[Steven Gill <st...@gmail.com>]

Woohoo! Thanks for the update Michal! And thanks to everyone who worked on
this!


On Fri, Aug 15, 2014 at 1:32 PM, Michal Mocny <mm...@google.com> wrote:

> ..and although it was mentioned in another thread, I may as well add to the
> summary: CI is in bad shape at the moment and Max is away for a week so it
> will stay like that for a bit.
>
> Sergey has done some cool work to make Medic easier to set up that Max will
> take a look at when he gets back, and Sergey will also be adding windows
> tests to the waterfall soon!
>
> (Then all we will need is directed emails for breaking the build)
>
> -Michal
>
>
> On Fri, Aug 15, 2014 at 4:28 PM, Michal Mocny <mm...@google.com> wrote:
>
> > Thanks a bunch to Stacy & Martin (& Jesse) for knocking back a lot of the
> > work to land new-style plugin tests in the last weeks.
> >
> > I've done some final cleanup this week, and think that we are for the
> > large part finally in the new world!  Woo hoo!
> >
> >
> > A few questions:
> > - Since many new-style tests landed, and old ones were removed, I went
> > ahead and closed off the JIRA issues.  If there is outstanding work to be
> > done, we can reopen.  I assume there isn't, because that would imply that
> > there are tests that have gone temporarily missing from any flow (bad!).
> >
> > - I noticed during cleanup that mobile-spec was never running whitelist
> > tests as part of the auto test suite.  So I fixed the bug.  Except
> > whitelist tests don't all pass on Android.  Were those test purposely not
> > being run?
> >
> > Clearly the current test-harness app is ugly and needs work, and Jason
> > (the newest member over at Google) has volunteered to pick up work on
> > adding a few improvements here as he ramps up on cordova stuff.
> >
> > -Michal
> >
>

Re: Update on Mobile Spec

[Michal Mocny <mm...@google.com>]

..and although it was mentioned in another thread, I may as well add to the
summary: CI is in bad shape at the moment and Max is away for a week so it
will stay like that for a bit.

Sergey has done some cool work to make Medic easier to set up that Max will
take a look at when he gets back, and Sergey will also be adding windows
tests to the waterfall soon!

(Then all we will need is directed emails for breaking the build)

-Michal


On Fri, Aug 15, 2014 at 4:28 PM, Michal Mocny <mm...@google.com> wrote:

> Thanks a bunch to Stacy & Martin (& Jesse) for knocking back a lot of the
> work to land new-style plugin tests in the last weeks.
>
> I've done some final cleanup this week, and think that we are for the
> large part finally in the new world!  Woo hoo!
>
>
> A few questions:
> - Since many new-style tests landed, and old ones were removed, I went
> ahead and closed off the JIRA issues.  If there is outstanding work to be
> done, we can reopen.  I assume there isn't, because that would imply that
> there are tests that have gone temporarily missing from any flow (bad!).
>
> - I noticed during cleanup that mobile-spec was never running whitelist
> tests as part of the auto test suite.  So I fixed the bug.  Except
> whitelist tests don't all pass on Android.  Were those test purposely not
> being run?
>
> Clearly the current test-harness app is ugly and needs work, and Jason
> (the newest member over at Google) has volunteered to pick up work on
> adding a few improvements here as he ramps up on cordova stuff.
>
> -Michal
>

Re: Update on Mobile Spec

[Michal Mocny <mm...@google.com>]

Thanks Ian. I will remove it from the default list again. Can't do it right
at this moment.
On 15 Aug 2014 23:31, "Ian Clelland" <ic...@chromium.org> wrote:

> On Fri, Aug 15, 2014 at 4:28 PM, Michal Mocny <mm...@google.com> wrote:
>
> > - I noticed during cleanup that mobile-spec was never running whitelist
> > tests as part of the auto test suite.  So I fixed the bug.  Except
> > whitelist tests don't all pass on Android.  Were those test purposely not
> > being run?
> >
>
> That was a deliberate decision -- see CB-4093 for background -- the tests
> are in mobilspec because it's the easiest place to run them, but at the
> time that I added them, they would absolutely not pass on any other
> platform than iOS and Android. (Because of the lack of the dedicated
> testing plugin, if not the actual whitelist implementation)
>
> Rather than deliberately add 20 or 30 failing tests to every platform's
> mobilespec, especially on platforms with no interest (or possibility
> ability) to implement the whitelist in exactly the same way, I decided to
> add it but not make it part of the "Run all tests" suite.
>
> (Of course, not being run by CI on every commit meant that those tests
> weren't being run in practice, so I'm not surprised that one ended up
> failing -- I'll take a look at that one)
>
> Ian
>

Re: Update on Mobile Spec

[Ian Clelland <ic...@chromium.org>]

On Fri, Aug 15, 2014 at 4:28 PM, Michal Mocny <mm...@google.com> wrote:

> - I noticed during cleanup that mobile-spec was never running whitelist
> tests as part of the auto test suite.  So I fixed the bug.  Except
> whitelist tests don't all pass on Android.  Were those test purposely not
> being run?
>

That was a deliberate decision -- see CB-4093 for background -- the tests
are in mobilspec because it's the easiest place to run them, but at the
time that I added them, they would absolutely not pass on any other
platform than iOS and Android. (Because of the lack of the dedicated
testing plugin, if not the actual whitelist implementation)

Rather than deliberately add 20 or 30 failing tests to every platform's
mobilespec, especially on platforms with no interest (or possibility
ability) to implement the whitelist in exactly the same way, I decided to
add it but not make it part of the "Run all tests" suite.

(Of course, not being run by CI on every commit meant that those tests
weren't being run in practice, so I'm not surprised that one ended up
failing -- I'll take a look at that one)

Ian