Failures in Arrow tests run outside of continuous integration

[Wes McKinney <we...@gmail.com>]

hi folks,

We have a number of integration tests and extra-CI build
configurations that are run via Crossbow, and many of them are
failing:

https://github.com/ursa-labs/crossbow/branches/all?page=1&query=nightly-480&utf8=%E2%9C%93

At some point we would like to notify the dev@ list about these
failures when they happen (or in a nightly e-mail), but in the
meantime some of these probably should be fixed (e.g. the Spark
integration build is failing) before the project can release again

Thanks
Wes

Re: Failures in Arrow tests run outside of continuous integration

[Wes McKinney <we...@gmail.com>]

The Docker nightlies are almost back to passing

https://github.com/ursa-labs/crossbow/branches/all?page=1&query=nightly-484&utf8=%E2%9C%93

I put up a WIP patch for the c_glib build but will need help with
that, and a patch for R. On the others

* I will look into turbodbc to see if it can be fixed easily, it
requires an upstream patch that hasn't been merged yet
* If someone could look into the Dask Docker build that would be great

- Wes

On Thu, Jun 20, 2019 at 7:42 AM Francois Saint-Jacques
<fs...@gmail.com> wrote:
>
> Ubuntu 16.04 ships with either 3.4 or 3.5
> Centos 6 with epel ships with 3.4, with scl it's 3.5 or 3.6
>
> I'd say keep just for ub16.
>
> François

Re: Failures in Arrow tests run outside of continuous integration

[Francois Saint-Jacques <fs...@gmail.com>]

Ubuntu 16.04 ships with either 3.4 or 3.5
Centos 6 with epel ships with 3.4, with scl it's 3.5 or 3.6

I'd say keep just for ub16.

François

Re: Failures in Arrow tests run outside of continuous integration

[Antoine Pitrou <so...@pitrou.net>]

Note: we could drop Python 3.5 there... What do people think?



On Thu, 20 Jun 2019 11:20:21 +0200
Krisztián Szűcs <sz...@gmail.com> wrote:
> These are the packaging tasks, the red ones are blocking the release:
> https://github.com/ursa-labs/crossbow/branches/all?query=nightly-479
> 
> The conda tasks should be ported to pipelines, because the travis builds
> simply stopped working (without saying anything to either stderr or stdout)
> and conda smithy now uses pipelines with more maintainable configurations:
> https://github.com/conda-forge/arrow-cpp-feedstock/tree/master/.azure-pipelines
> 
> On Thu, Jun 20, 2019 at 11:16 AM Antoine Pitrou <an...@python.org> wrote:
> 
> >
> > Le 20/06/2019 à 11:14, Krisztián Szűcs a écrit :  
> > > Travis provides 50 minutes timeout for free builds [1], so We need to run
> > > those
> > > tests either on ursabot or another hosted CI service. With crossbow we're
> > > not
> > > limited to certain CI services, so we can use CircleCI and Azure  
> > Pipelines  
> > > too,
> > > and simultaneously.
> > > Created issue https://issues.apache.org/jira/browse/ARROW-5664, I'm  
> > handling  
> > > this. Other tasks still need help though.  
> >
> > What are "other tasks"?
> >
> >  
> > >
> > > [1]:  
> > https://docs.travis-ci.com/user/customizing-the-build#build-timeouts  
> > > [2]: https://circleci.com/docs/2.0/configuration-reference/#jobs
> > >
> > > On Wed, Jun 19, 2019 at 7:09 PM Bryan Cutler <cu...@gmail.com> wrote:
> > >  
> > >> The last Spark Integration test timed out. Is there any way to increase  
> > the  
> > >> timeout for Crossbow tests?
> > >>
> > >> On Wed, Jun 19, 2019 at 10:05 AM Wes McKinney <we...@gmail.com>  
> > wrote:  
> > >>  
> > >>> hi folks,
> > >>>
> > >>> We have a number of integration tests and extra-CI build
> > >>> configurations that are run via Crossbow, and many of them are
> > >>> failing:
> > >>>
> > >>>
> > >>>  
> > >>  
> > https://github.com/ursa-labs/crossbow/branches/all?page=1&query=nightly-480&utf8=%E2%9C%93  
> > >>>
> > >>> At some point we would like to notify the dev@ list about these
> > >>> failures when they happen (or in a nightly e-mail), but in the
> > >>> meantime some of these probably should be fixed (e.g. the Spark
> > >>> integration build is failing) before the project can release again
> > >>>
> > >>> Thanks
> > >>> Wes
> > >>>  
> > >>  
> > >  
> >  
> 

Re: Failures in Arrow tests run outside of continuous integration

[Krisztián Szűcs <sz...@gmail.com>]

These are the packaging tasks, the red ones are blocking the release:
https://github.com/ursa-labs/crossbow/branches/all?query=nightly-479

The conda tasks should be ported to pipelines, because the travis builds
simply stopped working (without saying anything to either stderr or stdout)
and conda smithy now uses pipelines with more maintainable configurations:
https://github.com/conda-forge/arrow-cpp-feedstock/tree/master/.azure-pipelines

On Thu, Jun 20, 2019 at 11:16 AM Antoine Pitrou <an...@python.org> wrote:

>
> Le 20/06/2019 à 11:14, Krisztián Szűcs a écrit :
> > Travis provides 50 minutes timeout for free builds [1], so We need to run
> > those
> > tests either on ursabot or another hosted CI service. With crossbow we're
> > not
> > limited to certain CI services, so we can use CircleCI and Azure
> Pipelines
> > too,
> > and simultaneously.
> > Created issue https://issues.apache.org/jira/browse/ARROW-5664, I'm
> handling
> > this. Other tasks still need help though.
>
> What are "other tasks"?
>
>
> >
> > [1]:
> https://docs.travis-ci.com/user/customizing-the-build#build-timeouts
> > [2]: https://circleci.com/docs/2.0/configuration-reference/#jobs
> >
> > On Wed, Jun 19, 2019 at 7:09 PM Bryan Cutler <cu...@gmail.com> wrote:
> >
> >> The last Spark Integration test timed out. Is there any way to increase
> the
> >> timeout for Crossbow tests?
> >>
> >> On Wed, Jun 19, 2019 at 10:05 AM Wes McKinney <we...@gmail.com>
> wrote:
> >>
> >>> hi folks,
> >>>
> >>> We have a number of integration tests and extra-CI build
> >>> configurations that are run via Crossbow, and many of them are
> >>> failing:
> >>>
> >>>
> >>>
> >>
> https://github.com/ursa-labs/crossbow/branches/all?page=1&query=nightly-480&utf8=%E2%9C%93
> >>>
> >>> At some point we would like to notify the dev@ list about these
> >>> failures when they happen (or in a nightly e-mail), but in the
> >>> meantime some of these probably should be fixed (e.g. the Spark
> >>> integration build is failing) before the project can release again
> >>>
> >>> Thanks
> >>> Wes
> >>>
> >>
> >
>

Re: Failures in Arrow tests run outside of continuous integration

[Antoine Pitrou <an...@python.org>]

Le 20/06/2019 à 11:14, Krisztián Szűcs a écrit :
> Travis provides 50 minutes timeout for free builds [1], so We need to run
> those
> tests either on ursabot or another hosted CI service. With crossbow we're
> not
> limited to certain CI services, so we can use CircleCI and Azure Pipelines
> too,
> and simultaneously.
> Created issue https://issues.apache.org/jira/browse/ARROW-5664, I'm handling
> this. Other tasks still need help though.

What are "other tasks"?


> 
> [1]: https://docs.travis-ci.com/user/customizing-the-build#build-timeouts
> [2]: https://circleci.com/docs/2.0/configuration-reference/#jobs
> 
> On Wed, Jun 19, 2019 at 7:09 PM Bryan Cutler <cu...@gmail.com> wrote:
> 
>> The last Spark Integration test timed out. Is there any way to increase the
>> timeout for Crossbow tests?
>>
>> On Wed, Jun 19, 2019 at 10:05 AM Wes McKinney <we...@gmail.com> wrote:
>>
>>> hi folks,
>>>
>>> We have a number of integration tests and extra-CI build
>>> configurations that are run via Crossbow, and many of them are
>>> failing:
>>>
>>>
>>>
>> https://github.com/ursa-labs/crossbow/branches/all?page=1&query=nightly-480&utf8=%E2%9C%93
>>>
>>> At some point we would like to notify the dev@ list about these
>>> failures when they happen (or in a nightly e-mail), but in the
>>> meantime some of these probably should be fixed (e.g. the Spark
>>> integration build is failing) before the project can release again
>>>
>>> Thanks
>>> Wes
>>>
>>
> 

Re: Failures in Arrow tests run outside of continuous integration

[Krisztián Szűcs <sz...@gmail.com>]

Travis provides 50 minutes timeout for free builds [1], so We need to run
those
tests either on ursabot or another hosted CI service. With crossbow we're
not
limited to certain CI services, so we can use CircleCI and Azure Pipelines
too,
and simultaneously.
Created issue https://issues.apache.org/jira/browse/ARROW-5664, I'm handling
this. Other tasks still need help though.

[1]: https://docs.travis-ci.com/user/customizing-the-build#build-timeouts
[2]: https://circleci.com/docs/2.0/configuration-reference/#jobs

On Wed, Jun 19, 2019 at 7:09 PM Bryan Cutler <cu...@gmail.com> wrote:

> The last Spark Integration test timed out. Is there any way to increase the
> timeout for Crossbow tests?
>
> On Wed, Jun 19, 2019 at 10:05 AM Wes McKinney <we...@gmail.com> wrote:
>
> > hi folks,
> >
> > We have a number of integration tests and extra-CI build
> > configurations that are run via Crossbow, and many of them are
> > failing:
> >
> >
> >
> https://github.com/ursa-labs/crossbow/branches/all?page=1&query=nightly-480&utf8=%E2%9C%93
> >
> > At some point we would like to notify the dev@ list about these
> > failures when they happen (or in a nightly e-mail), but in the
> > meantime some of these probably should be fixed (e.g. the Spark
> > integration build is failing) before the project can release again
> >
> > Thanks
> > Wes
> >
>

Re: Failures in Arrow tests run outside of continuous integration

[Bryan Cutler <cu...@gmail.com>]

The last Spark Integration test timed out. Is there any way to increase the
timeout for Crossbow tests?

On Wed, Jun 19, 2019 at 10:05 AM Wes McKinney <we...@gmail.com> wrote:

> hi folks,
>
> We have a number of integration tests and extra-CI build
> configurations that are run via Crossbow, and many of them are
> failing:
>
>
> https://github.com/ursa-labs/crossbow/branches/all?page=1&query=nightly-480&utf8=%E2%9C%93
>
> At some point we would like to notify the dev@ list about these
> failures when they happen (or in a nightly e-mail), but in the
> meantime some of these probably should be fixed (e.g. the Spark
> integration build is failing) before the project can release again
>
> Thanks
> Wes
>