You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Jarek Potiuk <Ja...@polidea.com> on 2018/09/27 06:20:35 UTC
Travis CI tests failing in master
Hello Everyone,
Seems that since yesterday the builds started to fail in Travis CI badly.
And we need some urgent actions to fix it otherwise everyone developing
Airflow is affected now.
I am a bit fresh in Airflow so I am not sure how to handle those problems
in an "emergency" way (and I am not sure if I am the best to propose
solutions), so it would be great that more experienced people could help to
solve it.
It was not caused by any change - builds that worked before, started to
fail now. The nature of those errors suggests transitive dependencies
problems (they are all errors during installation of requirements).
Currently the master of airflow-incubator is broken. Actually the problem
has worsened - now we seem to have two distinct problems depending on
python requirement.
*Error 1: Flask appbuilder*
Yesterday we had just this one (for both python 2.7 and 3.5):
"flask-appbuilder 1.11.1 has requirement click==6.7, but you'll have click
7.0 which is incompatible."
and resulting error:
pkg_resources.DistributionNotFound: The 'click==6.7' distribution was not
found and is required by flask-appbuilder
You can see it failing in the master here:
https://travis-ci.org/apache/incubator-airflow/builds/433628053
Likely the solution is to pin flask-appbuilder to one of the earlier
versions or exclude 1.11.1 from valid versions, or limit flask-appbuilder
to <1.11.1 (but I am not sure which one is the best solution). I will try
some of that soon to see if that helps and let you know.
*Error 2: pynacl build problem *
This is only affecting python3.5. And started to appear only today in new
pull requests (of several people including mine, so it's again some
dependency problem).
Exception: ERROR: The 'make' utility is missing from PATH
----------------------------------------
Failed building wheel for pynacl
Running setup.py clean for pynacl
One of the failing builds is here:
https://travis-ci.org/apache/incubator-airflow/jobs/433920190
That one is tougher and seems to be related to
https://github.com/pyca/pynacl/issues/299.. This one is a bit tricky
and there is no obvious solution - maybe we should limit pynacl to some
earlier (which?) version ? Can anyone help with that ?
J.
--
*Jarek Potiuk, Principal Software Engineer*
Mobile: +48 660 796 129
Re: Travis CI tests failing in master
Posted by Jarek Potiuk <Ja...@polidea.com>.
Thanks everyone for quick resolve! I can confirm it also works for us now
and we are unblocked with sending our first PR very soon :).
J.
On Fri, Sep 28, 2018 at 12:13 PM Kaxil Naik <ka...@gmail.com> wrote:
> Haha. Well, what can I say? The solution was changing `pip` tp `pip2`.
> Banging my head against the wall. I just read yesterday that it wasn't
> working and now 😁... well well...
>
> On Fri, Sep 28, 2018 at 10:02 AM Ash Berlin-Taylor <as...@apache.org> wrote:
>
> > Looks like someone has beaten you to the punch
> >
> https://github.com/apache/incubator-airflow/pull/3968#pullrequestreview-159772413
> > <
> >
> https://github.com/apache/incubator-airflow/pull/3968#pullrequestreview-159772413
> > >
> >
> > :)
> >
> > -ash
> >
> > > On 28 Sep 2018, at 09:59, Kaxil Naik <ka...@gmail.com> wrote:
> > >
> > > The problem is 80% solved. The only failing error are related to
> > > TestPythonVirtualenvOperator .
> > > More details are discussed at
> > > https://github.com/apache/incubator-airflow-ci/pull/3
> > >
> > > It is because protocol=3 is used instead of 2.
> > >
> > > Can someone take a dig to resolve this?
> > >
> > > Regards,
> > > Kaxil
> > >
> > >
> > > On Thu, Sep 27, 2018 at 11:36 AM Jarek Potiuk <
> Jarek.Potiuk@polidea.com>
> > > wrote:
> > >
> > >> I see also that the solution is on it's way for the first problem as
> > well
> > >> :). https://github.com/apache/incubator-airflow/pull/3962
> > >>
> > >> On Thu, Sep 27, 2018 at 10:47 AM Jarek Potiuk <
> Jarek.Potiuk@polidea.com
> > >
> > >> wrote:
> > >>
> > >>> Seem that the second problem is already solved in this PR:
> > >>> https://github.com/apache/incubator-airflow-ci/pull/3 - hopefully
> will
> > >> be
> > >>> merged soon :).
> > >>>
> > >>> On Thu, Sep 27, 2018 at 9:57 AM airflowuser
> > >>> <ai...@protonmail.com.invalid> wrote:
> > >>>
> > >>>> Also I see this a lot:
> > >>>> 1) ERROR: Failure: ProgrammingError
> > >> ((_mysql_exceptions.ProgrammingError)
> > >>>> (1146, "Table 'airflow.task_instance' doesn't exist") [SQL: u'DELETE
> > >> FROM
> > >>>> task_instance WHERE task_instance.dag_id = %s'] [parameters:
> > >>>> ('unit_tests',)])
> > >>>>
> > >>>>
> > >>>> Sent with ProtonMail Secure Email.
> > >>>>
> > >>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > >>>> On Thursday, September 27, 2018 9:20 AM, Jarek Potiuk <
> > >>>> Jarek.Potiuk@polidea.com> wrote:
> > >>>>
> > >>>>> Hello Everyone,
> > >>>>>
> > >>>>> Seems that since yesterday the builds started to fail in Travis CI
> > >>>> badly.
> > >>>>> And we need some urgent actions to fix it otherwise everyone
> > >> developing
> > >>>>> Airflow is affected now.
> > >>>>>
> > >>>>> I am a bit fresh in Airflow so I am not sure how to handle those
> > >>>> problems
> > >>>>> in an "emergency" way (and I am not sure if I am the best to
> propose
> > >>>>> solutions), so it would be great that more experienced people could
> > >>>> help to
> > >>>>> solve it.
> > >>>>>
> > >>>>> It was not caused by any change - builds that worked before,
> started
> > >> to
> > >>>>> fail now. The nature of those errors suggests transitive
> dependencies
> > >>>>> problems (they are all errors during installation of requirements).
> > >>>>>
> > >>>>> Currently the master of airflow-incubator is broken. Actually the
> > >>>> problem
> > >>>>> has worsened - now we seem to have two distinct problems depending
> on
> > >>>>> python requirement.
> > >>>>>
> > >>>>> Error 1: Flask appbuilder
> > >>>>>
> > >>>>> Yesterday we had just this one (for both python 2.7 and 3.5):
> > >>>>> "flask-appbuilder 1.11.1 has requirement click==6.7, but you'll
> have
> > >>>> click
> > >>>>> 7.0 which is incompatible."
> > >>>>> and resulting error:
> > >>>>>
> > >>>>> pkg_resources.DistributionNotFound: The 'click==6.7' distribution
> was
> > >>>> not
> > >>>>> found and is required by flask-appbuilder
> > >>>>>
> > >>>>> You can see it failing in the master here:
> > >>>>> https://travis-ci.org/apache/incubator-airflow/builds/433628053
> > >>>>>
> > >>>>> Likely the solution is to pin flask-appbuilder to one of the
> earlier
> > >>>>> versions or exclude 1.11.1 from valid versions, or limit
> > >>>> flask-appbuilder
> > >>>>> to <1.11.1 (but I am not sure which one is the best solution). I
> will
> > >>>> try
> > >>>>> some of that soon to see if that helps and let you know.
> > >>>>>
> > >>>>> *Error 2: pynacl build problem *
> > >>>>>
> > >>>>> This is only affecting python3.5. And started to appear only today
> in
> > >>>> new
> > >>>>> pull requests (of several people including mine, so it's again some
> > >>>>> dependency problem).
> > >>>>>
> > >>>>> Exception: ERROR: The 'make' utility is missing from PATH
> > >>>>>
> > >>>>>
> > >>>>
> > >>
> >
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > >>>>>
> > >>>>> Failed building wheel for pynacl
> > >>>>> Running setup.py clean for pynacl
> > >>>>>
> > >>>>> One of the failing builds is here:
> > >>>>> https://travis-ci.org/apache/incubator-airflow/jobs/433920190
> > >>>>>
> > >>>>> That one is tougher and seems to be related to
> > >>>>> https://github.com/pyca/pynacl/issues/299.. This one is a bit
> tricky
> > >>>>> and there is no obvious solution - maybe we should limit pynacl to
> > >> some
> > >>>>> earlier (which?) version ? Can anyone help with that ?
> > >>>>>
> > >>>>> J.
> > >>>>>
> > >>>>>
> > >>>>
> > >>
> >
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> > >>>>>
> > >>>>> Jarek Potiuk, Principal Software Engineer
> > >>>>> Mobile: +48 660 796 129
> > >>>>
> > >>>>
> > >>>>
> > >>>
> > >>> --
> > >>>
> > >>> *Jarek Potiuk, Principal Software Engineer*
> > >>> Mobile: +48 660 796 129
> > >>>
> > >>
> > >>
> > >> --
> > >>
> > >> *Jarek Potiuk, Principal Software Engineer*
> > >> Mobile: +48 660 796 129
> > >>
> > >
> > >
> > > --
> > > *Kaxil Naik*
> > > *Big Data Consultant *@ *Data Reply UK*
> > > *Certified *Google Cloud Data Engineer | *Certified* Apache Spark &
> Neo4j
> > > Developer
> > > *Phone: *+44 (0) 74820 88992
> > > *LinkedIn*: https://www.linkedin.com/in/kaxil
> >
> >
>
> --
> *Kaxil Naik*
> *Big Data Consultant *@ *Data Reply UK*
> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
> Developer
> *Phone: *+44 (0) 74820 88992
> *LinkedIn*: https://www.linkedin.com/in/kaxil
>
--
*Jarek Potiuk, Principal Software Engineer*
Mobile: +48 660 796 129
Re: Travis CI tests failing in master
Posted by Kaxil Naik <ka...@gmail.com>.
Haha. Well, what can I say? The solution was changing `pip` tp `pip2`.
Banging my head against the wall. I just read yesterday that it wasn't
working and now 😁... well well...
On Fri, Sep 28, 2018 at 10:02 AM Ash Berlin-Taylor <as...@apache.org> wrote:
> Looks like someone has beaten you to the punch
> https://github.com/apache/incubator-airflow/pull/3968#pullrequestreview-159772413
> <
> https://github.com/apache/incubator-airflow/pull/3968#pullrequestreview-159772413
> >
>
> :)
>
> -ash
>
> > On 28 Sep 2018, at 09:59, Kaxil Naik <ka...@gmail.com> wrote:
> >
> > The problem is 80% solved. The only failing error are related to
> > TestPythonVirtualenvOperator .
> > More details are discussed at
> > https://github.com/apache/incubator-airflow-ci/pull/3
> >
> > It is because protocol=3 is used instead of 2.
> >
> > Can someone take a dig to resolve this?
> >
> > Regards,
> > Kaxil
> >
> >
> > On Thu, Sep 27, 2018 at 11:36 AM Jarek Potiuk <Ja...@polidea.com>
> > wrote:
> >
> >> I see also that the solution is on it's way for the first problem as
> well
> >> :). https://github.com/apache/incubator-airflow/pull/3962
> >>
> >> On Thu, Sep 27, 2018 at 10:47 AM Jarek Potiuk <Jarek.Potiuk@polidea.com
> >
> >> wrote:
> >>
> >>> Seem that the second problem is already solved in this PR:
> >>> https://github.com/apache/incubator-airflow-ci/pull/3 - hopefully will
> >> be
> >>> merged soon :).
> >>>
> >>> On Thu, Sep 27, 2018 at 9:57 AM airflowuser
> >>> <ai...@protonmail.com.invalid> wrote:
> >>>
> >>>> Also I see this a lot:
> >>>> 1) ERROR: Failure: ProgrammingError
> >> ((_mysql_exceptions.ProgrammingError)
> >>>> (1146, "Table 'airflow.task_instance' doesn't exist") [SQL: u'DELETE
> >> FROM
> >>>> task_instance WHERE task_instance.dag_id = %s'] [parameters:
> >>>> ('unit_tests',)])
> >>>>
> >>>>
> >>>> Sent with ProtonMail Secure Email.
> >>>>
> >>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> >>>> On Thursday, September 27, 2018 9:20 AM, Jarek Potiuk <
> >>>> Jarek.Potiuk@polidea.com> wrote:
> >>>>
> >>>>> Hello Everyone,
> >>>>>
> >>>>> Seems that since yesterday the builds started to fail in Travis CI
> >>>> badly.
> >>>>> And we need some urgent actions to fix it otherwise everyone
> >> developing
> >>>>> Airflow is affected now.
> >>>>>
> >>>>> I am a bit fresh in Airflow so I am not sure how to handle those
> >>>> problems
> >>>>> in an "emergency" way (and I am not sure if I am the best to propose
> >>>>> solutions), so it would be great that more experienced people could
> >>>> help to
> >>>>> solve it.
> >>>>>
> >>>>> It was not caused by any change - builds that worked before, started
> >> to
> >>>>> fail now. The nature of those errors suggests transitive dependencies
> >>>>> problems (they are all errors during installation of requirements).
> >>>>>
> >>>>> Currently the master of airflow-incubator is broken. Actually the
> >>>> problem
> >>>>> has worsened - now we seem to have two distinct problems depending on
> >>>>> python requirement.
> >>>>>
> >>>>> Error 1: Flask appbuilder
> >>>>>
> >>>>> Yesterday we had just this one (for both python 2.7 and 3.5):
> >>>>> "flask-appbuilder 1.11.1 has requirement click==6.7, but you'll have
> >>>> click
> >>>>> 7.0 which is incompatible."
> >>>>> and resulting error:
> >>>>>
> >>>>> pkg_resources.DistributionNotFound: The 'click==6.7' distribution was
> >>>> not
> >>>>> found and is required by flask-appbuilder
> >>>>>
> >>>>> You can see it failing in the master here:
> >>>>> https://travis-ci.org/apache/incubator-airflow/builds/433628053
> >>>>>
> >>>>> Likely the solution is to pin flask-appbuilder to one of the earlier
> >>>>> versions or exclude 1.11.1 from valid versions, or limit
> >>>> flask-appbuilder
> >>>>> to <1.11.1 (but I am not sure which one is the best solution). I will
> >>>> try
> >>>>> some of that soon to see if that helps and let you know.
> >>>>>
> >>>>> *Error 2: pynacl build problem *
> >>>>>
> >>>>> This is only affecting python3.5. And started to appear only today in
> >>>> new
> >>>>> pull requests (of several people including mine, so it's again some
> >>>>> dependency problem).
> >>>>>
> >>>>> Exception: ERROR: The 'make' utility is missing from PATH
> >>>>>
> >>>>>
> >>>>
> >>
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >>>>>
> >>>>> Failed building wheel for pynacl
> >>>>> Running setup.py clean for pynacl
> >>>>>
> >>>>> One of the failing builds is here:
> >>>>> https://travis-ci.org/apache/incubator-airflow/jobs/433920190
> >>>>>
> >>>>> That one is tougher and seems to be related to
> >>>>> https://github.com/pyca/pynacl/issues/299.. This one is a bit tricky
> >>>>> and there is no obvious solution - maybe we should limit pynacl to
> >> some
> >>>>> earlier (which?) version ? Can anyone help with that ?
> >>>>>
> >>>>> J.
> >>>>>
> >>>>>
> >>>>
> >>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >>>>>
> >>>>> Jarek Potiuk, Principal Software Engineer
> >>>>> Mobile: +48 660 796 129
> >>>>
> >>>>
> >>>>
> >>>
> >>> --
> >>>
> >>> *Jarek Potiuk, Principal Software Engineer*
> >>> Mobile: +48 660 796 129
> >>>
> >>
> >>
> >> --
> >>
> >> *Jarek Potiuk, Principal Software Engineer*
> >> Mobile: +48 660 796 129
> >>
> >
> >
> > --
> > *Kaxil Naik*
> > *Big Data Consultant *@ *Data Reply UK*
> > *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
> > Developer
> > *Phone: *+44 (0) 74820 88992
> > *LinkedIn*: https://www.linkedin.com/in/kaxil
>
>
--
*Kaxil Naik*
*Big Data Consultant *@ *Data Reply UK*
*Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
Developer
*Phone: *+44 (0) 74820 88992
*LinkedIn*: https://www.linkedin.com/in/kaxil
Re: Travis CI tests failing in master
Posted by Ash Berlin-Taylor <as...@apache.org>.
Looks like someone has beaten you to the punch https://github.com/apache/incubator-airflow/pull/3968#pullrequestreview-159772413 <https://github.com/apache/incubator-airflow/pull/3968#pullrequestreview-159772413>
:)
-ash
> On 28 Sep 2018, at 09:59, Kaxil Naik <ka...@gmail.com> wrote:
>
> The problem is 80% solved. The only failing error are related to
> TestPythonVirtualenvOperator .
> More details are discussed at
> https://github.com/apache/incubator-airflow-ci/pull/3
>
> It is because protocol=3 is used instead of 2.
>
> Can someone take a dig to resolve this?
>
> Regards,
> Kaxil
>
>
> On Thu, Sep 27, 2018 at 11:36 AM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
>> I see also that the solution is on it's way for the first problem as well
>> :). https://github.com/apache/incubator-airflow/pull/3962
>>
>> On Thu, Sep 27, 2018 at 10:47 AM Jarek Potiuk <Ja...@polidea.com>
>> wrote:
>>
>>> Seem that the second problem is already solved in this PR:
>>> https://github.com/apache/incubator-airflow-ci/pull/3 - hopefully will
>> be
>>> merged soon :).
>>>
>>> On Thu, Sep 27, 2018 at 9:57 AM airflowuser
>>> <ai...@protonmail.com.invalid> wrote:
>>>
>>>> Also I see this a lot:
>>>> 1) ERROR: Failure: ProgrammingError
>> ((_mysql_exceptions.ProgrammingError)
>>>> (1146, "Table 'airflow.task_instance' doesn't exist") [SQL: u'DELETE
>> FROM
>>>> task_instance WHERE task_instance.dag_id = %s'] [parameters:
>>>> ('unit_tests',)])
>>>>
>>>>
>>>> Sent with ProtonMail Secure Email.
>>>>
>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>>> On Thursday, September 27, 2018 9:20 AM, Jarek Potiuk <
>>>> Jarek.Potiuk@polidea.com> wrote:
>>>>
>>>>> Hello Everyone,
>>>>>
>>>>> Seems that since yesterday the builds started to fail in Travis CI
>>>> badly.
>>>>> And we need some urgent actions to fix it otherwise everyone
>> developing
>>>>> Airflow is affected now.
>>>>>
>>>>> I am a bit fresh in Airflow so I am not sure how to handle those
>>>> problems
>>>>> in an "emergency" way (and I am not sure if I am the best to propose
>>>>> solutions), so it would be great that more experienced people could
>>>> help to
>>>>> solve it.
>>>>>
>>>>> It was not caused by any change - builds that worked before, started
>> to
>>>>> fail now. The nature of those errors suggests transitive dependencies
>>>>> problems (they are all errors during installation of requirements).
>>>>>
>>>>> Currently the master of airflow-incubator is broken. Actually the
>>>> problem
>>>>> has worsened - now we seem to have two distinct problems depending on
>>>>> python requirement.
>>>>>
>>>>> Error 1: Flask appbuilder
>>>>>
>>>>> Yesterday we had just this one (for both python 2.7 and 3.5):
>>>>> "flask-appbuilder 1.11.1 has requirement click==6.7, but you'll have
>>>> click
>>>>> 7.0 which is incompatible."
>>>>> and resulting error:
>>>>>
>>>>> pkg_resources.DistributionNotFound: The 'click==6.7' distribution was
>>>> not
>>>>> found and is required by flask-appbuilder
>>>>>
>>>>> You can see it failing in the master here:
>>>>> https://travis-ci.org/apache/incubator-airflow/builds/433628053
>>>>>
>>>>> Likely the solution is to pin flask-appbuilder to one of the earlier
>>>>> versions or exclude 1.11.1 from valid versions, or limit
>>>> flask-appbuilder
>>>>> to <1.11.1 (but I am not sure which one is the best solution). I will
>>>> try
>>>>> some of that soon to see if that helps and let you know.
>>>>>
>>>>> *Error 2: pynacl build problem *
>>>>>
>>>>> This is only affecting python3.5. And started to appear only today in
>>>> new
>>>>> pull requests (of several people including mine, so it's again some
>>>>> dependency problem).
>>>>>
>>>>> Exception: ERROR: The 'make' utility is missing from PATH
>>>>>
>>>>>
>>>>
>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>>>>
>>>>> Failed building wheel for pynacl
>>>>> Running setup.py clean for pynacl
>>>>>
>>>>> One of the failing builds is here:
>>>>> https://travis-ci.org/apache/incubator-airflow/jobs/433920190
>>>>>
>>>>> That one is tougher and seems to be related to
>>>>> https://github.com/pyca/pynacl/issues/299.. This one is a bit tricky
>>>>> and there is no obvious solution - maybe we should limit pynacl to
>> some
>>>>> earlier (which?) version ? Can anyone help with that ?
>>>>>
>>>>> J.
>>>>>
>>>>>
>>>>
>> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>>>>>
>>>>> Jarek Potiuk, Principal Software Engineer
>>>>> Mobile: +48 660 796 129
>>>>
>>>>
>>>>
>>>
>>> --
>>>
>>> *Jarek Potiuk, Principal Software Engineer*
>>> Mobile: +48 660 796 129
>>>
>>
>>
>> --
>>
>> *Jarek Potiuk, Principal Software Engineer*
>> Mobile: +48 660 796 129
>>
>
>
> --
> *Kaxil Naik*
> *Big Data Consultant *@ *Data Reply UK*
> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
> Developer
> *Phone: *+44 (0) 74820 88992
> *LinkedIn*: https://www.linkedin.com/in/kaxil
Re: Travis CI tests failing in master
Posted by Kaxil Naik <ka...@gmail.com>.
The problem is 80% solved. The only failing error are related to
TestPythonVirtualenvOperator .
More details are discussed at
https://github.com/apache/incubator-airflow-ci/pull/3
It is because protocol=3 is used instead of 2.
Can someone take a dig to resolve this?
Regards,
Kaxil
On Thu, Sep 27, 2018 at 11:36 AM Jarek Potiuk <Ja...@polidea.com>
wrote:
> I see also that the solution is on it's way for the first problem as well
> :). https://github.com/apache/incubator-airflow/pull/3962
>
> On Thu, Sep 27, 2018 at 10:47 AM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
> > Seem that the second problem is already solved in this PR:
> > https://github.com/apache/incubator-airflow-ci/pull/3 - hopefully will
> be
> > merged soon :).
> >
> > On Thu, Sep 27, 2018 at 9:57 AM airflowuser
> > <ai...@protonmail.com.invalid> wrote:
> >
> >> Also I see this a lot:
> >> 1) ERROR: Failure: ProgrammingError
> ((_mysql_exceptions.ProgrammingError)
> >> (1146, "Table 'airflow.task_instance' doesn't exist") [SQL: u'DELETE
> FROM
> >> task_instance WHERE task_instance.dag_id = %s'] [parameters:
> >> ('unit_tests',)])
> >>
> >>
> >> Sent with ProtonMail Secure Email.
> >>
> >> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> >> On Thursday, September 27, 2018 9:20 AM, Jarek Potiuk <
> >> Jarek.Potiuk@polidea.com> wrote:
> >>
> >> > Hello Everyone,
> >> >
> >> > Seems that since yesterday the builds started to fail in Travis CI
> >> badly.
> >> > And we need some urgent actions to fix it otherwise everyone
> developing
> >> > Airflow is affected now.
> >> >
> >> > I am a bit fresh in Airflow so I am not sure how to handle those
> >> problems
> >> > in an "emergency" way (and I am not sure if I am the best to propose
> >> > solutions), so it would be great that more experienced people could
> >> help to
> >> > solve it.
> >> >
> >> > It was not caused by any change - builds that worked before, started
> to
> >> > fail now. The nature of those errors suggests transitive dependencies
> >> > problems (they are all errors during installation of requirements).
> >> >
> >> > Currently the master of airflow-incubator is broken. Actually the
> >> problem
> >> > has worsened - now we seem to have two distinct problems depending on
> >> > python requirement.
> >> >
> >> > Error 1: Flask appbuilder
> >> >
> >> > Yesterday we had just this one (for both python 2.7 and 3.5):
> >> > "flask-appbuilder 1.11.1 has requirement click==6.7, but you'll have
> >> click
> >> > 7.0 which is incompatible."
> >> > and resulting error:
> >> >
> >> > pkg_resources.DistributionNotFound: The 'click==6.7' distribution was
> >> not
> >> > found and is required by flask-appbuilder
> >> >
> >> > You can see it failing in the master here:
> >> > https://travis-ci.org/apache/incubator-airflow/builds/433628053
> >> >
> >> > Likely the solution is to pin flask-appbuilder to one of the earlier
> >> > versions or exclude 1.11.1 from valid versions, or limit
> >> flask-appbuilder
> >> > to <1.11.1 (but I am not sure which one is the best solution). I will
> >> try
> >> > some of that soon to see if that helps and let you know.
> >> >
> >> > *Error 2: pynacl build problem *
> >> >
> >> > This is only affecting python3.5. And started to appear only today in
> >> new
> >> > pull requests (of several people including mine, so it's again some
> >> > dependency problem).
> >> >
> >> > Exception: ERROR: The 'make' utility is missing from PATH
> >> >
> >> >
> >>
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >> >
> >> > Failed building wheel for pynacl
> >> > Running setup.py clean for pynacl
> >> >
> >> > One of the failing builds is here:
> >> > https://travis-ci.org/apache/incubator-airflow/jobs/433920190
> >> >
> >> > That one is tougher and seems to be related to
> >> > https://github.com/pyca/pynacl/issues/299.. This one is a bit tricky
> >> > and there is no obvious solution - maybe we should limit pynacl to
> some
> >> > earlier (which?) version ? Can anyone help with that ?
> >> >
> >> > J.
> >> >
> >> >
> >>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >> >
> >> > Jarek Potiuk, Principal Software Engineer
> >> > Mobile: +48 660 796 129
> >>
> >>
> >>
> >
> > --
> >
> > *Jarek Potiuk, Principal Software Engineer*
> > Mobile: +48 660 796 129
> >
>
>
> --
>
> *Jarek Potiuk, Principal Software Engineer*
> Mobile: +48 660 796 129
>
--
*Kaxil Naik*
*Big Data Consultant *@ *Data Reply UK*
*Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
Developer
*Phone: *+44 (0) 74820 88992
*LinkedIn*: https://www.linkedin.com/in/kaxil
Re: Travis CI tests failing in master
Posted by Jarek Potiuk <Ja...@polidea.com>.
I see also that the solution is on it's way for the first problem as well
:). https://github.com/apache/incubator-airflow/pull/3962
On Thu, Sep 27, 2018 at 10:47 AM Jarek Potiuk <Ja...@polidea.com>
wrote:
> Seem that the second problem is already solved in this PR:
> https://github.com/apache/incubator-airflow-ci/pull/3 - hopefully will be
> merged soon :).
>
> On Thu, Sep 27, 2018 at 9:57 AM airflowuser
> <ai...@protonmail.com.invalid> wrote:
>
>> Also I see this a lot:
>> 1) ERROR: Failure: ProgrammingError ((_mysql_exceptions.ProgrammingError)
>> (1146, "Table 'airflow.task_instance' doesn't exist") [SQL: u'DELETE FROM
>> task_instance WHERE task_instance.dag_id = %s'] [parameters:
>> ('unit_tests',)])
>>
>>
>> Sent with ProtonMail Secure Email.
>>
>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>> On Thursday, September 27, 2018 9:20 AM, Jarek Potiuk <
>> Jarek.Potiuk@polidea.com> wrote:
>>
>> > Hello Everyone,
>> >
>> > Seems that since yesterday the builds started to fail in Travis CI
>> badly.
>> > And we need some urgent actions to fix it otherwise everyone developing
>> > Airflow is affected now.
>> >
>> > I am a bit fresh in Airflow so I am not sure how to handle those
>> problems
>> > in an "emergency" way (and I am not sure if I am the best to propose
>> > solutions), so it would be great that more experienced people could
>> help to
>> > solve it.
>> >
>> > It was not caused by any change - builds that worked before, started to
>> > fail now. The nature of those errors suggests transitive dependencies
>> > problems (they are all errors during installation of requirements).
>> >
>> > Currently the master of airflow-incubator is broken. Actually the
>> problem
>> > has worsened - now we seem to have two distinct problems depending on
>> > python requirement.
>> >
>> > Error 1: Flask appbuilder
>> >
>> > Yesterday we had just this one (for both python 2.7 and 3.5):
>> > "flask-appbuilder 1.11.1 has requirement click==6.7, but you'll have
>> click
>> > 7.0 which is incompatible."
>> > and resulting error:
>> >
>> > pkg_resources.DistributionNotFound: The 'click==6.7' distribution was
>> not
>> > found and is required by flask-appbuilder
>> >
>> > You can see it failing in the master here:
>> > https://travis-ci.org/apache/incubator-airflow/builds/433628053
>> >
>> > Likely the solution is to pin flask-appbuilder to one of the earlier
>> > versions or exclude 1.11.1 from valid versions, or limit
>> flask-appbuilder
>> > to <1.11.1 (but I am not sure which one is the best solution). I will
>> try
>> > some of that soon to see if that helps and let you know.
>> >
>> > *Error 2: pynacl build problem *
>> >
>> > This is only affecting python3.5. And started to appear only today in
>> new
>> > pull requests (of several people including mine, so it's again some
>> > dependency problem).
>> >
>> > Exception: ERROR: The 'make' utility is missing from PATH
>> >
>> >
>> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> >
>> > Failed building wheel for pynacl
>> > Running setup.py clean for pynacl
>> >
>> > One of the failing builds is here:
>> > https://travis-ci.org/apache/incubator-airflow/jobs/433920190
>> >
>> > That one is tougher and seems to be related to
>> > https://github.com/pyca/pynacl/issues/299.. This one is a bit tricky
>> > and there is no obvious solution - maybe we should limit pynacl to some
>> > earlier (which?) version ? Can anyone help with that ?
>> >
>> > J.
>> >
>> >
>> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>> >
>> > Jarek Potiuk, Principal Software Engineer
>> > Mobile: +48 660 796 129
>>
>>
>>
>
> --
>
> *Jarek Potiuk, Principal Software Engineer*
> Mobile: +48 660 796 129
>
--
*Jarek Potiuk, Principal Software Engineer*
Mobile: +48 660 796 129
Re: Travis CI tests failing in master
Posted by Jarek Potiuk <Ja...@polidea.com>.
Seem that the second problem is already solved in this PR:
https://github.com/apache/incubator-airflow-ci/pull/3 - hopefully will be
merged soon :).
On Thu, Sep 27, 2018 at 9:57 AM airflowuser
<ai...@protonmail.com.invalid> wrote:
> Also I see this a lot:
> 1) ERROR: Failure: ProgrammingError ((_mysql_exceptions.ProgrammingError)
> (1146, "Table 'airflow.task_instance' doesn't exist") [SQL: u'DELETE FROM
> task_instance WHERE task_instance.dag_id = %s'] [parameters:
> ('unit_tests',)])
>
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, September 27, 2018 9:20 AM, Jarek Potiuk <
> Jarek.Potiuk@polidea.com> wrote:
>
> > Hello Everyone,
> >
> > Seems that since yesterday the builds started to fail in Travis CI badly.
> > And we need some urgent actions to fix it otherwise everyone developing
> > Airflow is affected now.
> >
> > I am a bit fresh in Airflow so I am not sure how to handle those problems
> > in an "emergency" way (and I am not sure if I am the best to propose
> > solutions), so it would be great that more experienced people could help
> to
> > solve it.
> >
> > It was not caused by any change - builds that worked before, started to
> > fail now. The nature of those errors suggests transitive dependencies
> > problems (they are all errors during installation of requirements).
> >
> > Currently the master of airflow-incubator is broken. Actually the problem
> > has worsened - now we seem to have two distinct problems depending on
> > python requirement.
> >
> > Error 1: Flask appbuilder
> >
> > Yesterday we had just this one (for both python 2.7 and 3.5):
> > "flask-appbuilder 1.11.1 has requirement click==6.7, but you'll have
> click
> > 7.0 which is incompatible."
> > and resulting error:
> >
> > pkg_resources.DistributionNotFound: The 'click==6.7' distribution was not
> > found and is required by flask-appbuilder
> >
> > You can see it failing in the master here:
> > https://travis-ci.org/apache/incubator-airflow/builds/433628053
> >
> > Likely the solution is to pin flask-appbuilder to one of the earlier
> > versions or exclude 1.11.1 from valid versions, or limit flask-appbuilder
> > to <1.11.1 (but I am not sure which one is the best solution). I will try
> > some of that soon to see if that helps and let you know.
> >
> > *Error 2: pynacl build problem *
> >
> > This is only affecting python3.5. And started to appear only today in new
> > pull requests (of several people including mine, so it's again some
> > dependency problem).
> >
> > Exception: ERROR: The 'make' utility is missing from PATH
> >
> >
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >
> > Failed building wheel for pynacl
> > Running setup.py clean for pynacl
> >
> > One of the failing builds is here:
> > https://travis-ci.org/apache/incubator-airflow/jobs/433920190
> >
> > That one is tougher and seems to be related to
> > https://github.com/pyca/pynacl/issues/299.. This one is a bit tricky
> > and there is no obvious solution - maybe we should limit pynacl to some
> > earlier (which?) version ? Can anyone help with that ?
> >
> > J.
> >
> >
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >
> > Jarek Potiuk, Principal Software Engineer
> > Mobile: +48 660 796 129
>
>
>
--
*Jarek Potiuk, Principal Software Engineer*
Mobile: +48 660 796 129
Re: Travis CI tests failing in master
Posted by airflowuser <ai...@protonmail.com.INVALID>.
Also I see this a lot:
1) ERROR: Failure: ProgrammingError ((_mysql_exceptions.ProgrammingError) (1146, "Table 'airflow.task_instance' doesn't exist") [SQL: u'DELETE FROM task_instance WHERE task_instance.dag_id = %s'] [parameters: ('unit_tests',)])
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, September 27, 2018 9:20 AM, Jarek Potiuk <Ja...@polidea.com> wrote:
> Hello Everyone,
>
> Seems that since yesterday the builds started to fail in Travis CI badly.
> And we need some urgent actions to fix it otherwise everyone developing
> Airflow is affected now.
>
> I am a bit fresh in Airflow so I am not sure how to handle those problems
> in an "emergency" way (and I am not sure if I am the best to propose
> solutions), so it would be great that more experienced people could help to
> solve it.
>
> It was not caused by any change - builds that worked before, started to
> fail now. The nature of those errors suggests transitive dependencies
> problems (they are all errors during installation of requirements).
>
> Currently the master of airflow-incubator is broken. Actually the problem
> has worsened - now we seem to have two distinct problems depending on
> python requirement.
>
> Error 1: Flask appbuilder
>
> Yesterday we had just this one (for both python 2.7 and 3.5):
> "flask-appbuilder 1.11.1 has requirement click==6.7, but you'll have click
> 7.0 which is incompatible."
> and resulting error:
>
> pkg_resources.DistributionNotFound: The 'click==6.7' distribution was not
> found and is required by flask-appbuilder
>
> You can see it failing in the master here:
> https://travis-ci.org/apache/incubator-airflow/builds/433628053
>
> Likely the solution is to pin flask-appbuilder to one of the earlier
> versions or exclude 1.11.1 from valid versions, or limit flask-appbuilder
> to <1.11.1 (but I am not sure which one is the best solution). I will try
> some of that soon to see if that helps and let you know.
>
> *Error 2: pynacl build problem *
>
> This is only affecting python3.5. And started to appear only today in new
> pull requests (of several people including mine, so it's again some
> dependency problem).
>
> Exception: ERROR: The 'make' utility is missing from PATH
>
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Failed building wheel for pynacl
> Running setup.py clean for pynacl
>
> One of the failing builds is here:
> https://travis-ci.org/apache/incubator-airflow/jobs/433920190
>
> That one is tougher and seems to be related to
> https://github.com/pyca/pynacl/issues/299.. This one is a bit tricky
> and there is no obvious solution - maybe we should limit pynacl to some
> earlier (which?) version ? Can anyone help with that ?
>
> J.
>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Jarek Potiuk, Principal Software Engineer
> Mobile: +48 660 796 129