You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Chad Dombrova <ch...@gmail.com> on 2019/11/26 06:26:00 UTC
cython test instability
Hi all,
The cython tests started failing on one of my PRs which were succeeding
before. The error is one that I've never seen before (separated onto
different lines to make it easier to read):
Caused by: org.gradle.api.GradleException:
Could not copy file
'/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
/src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
to
'/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
/src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
Followed immediately by an error about could not create a directory of the
same name. Here's the gradle scan:
https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
Any ideas?
-chad
Re: cython test instability
Posted by Chad Dombrova <ch...@gmail.com>.
yeah, I've excised both test_requires and setup_requires in my test
simplification PR: https://github.com/apache/beam/pull/10038
I'm happy to see those go sooner rather than later, as it'll reduce the
scope of my PR. The rest of my PR is about ensuring that build
dependencies like cython and grpc are available at "build" time (i.e. when
setup.py gets called), and the modern solution for this is a
pep517/518-compliant build system, of which tox is one.
-chad
On Tue, Nov 26, 2019 at 6:39 PM Udi Meiri <eh...@google.com> wrote:
> I'm not sure about where the error with the simplegeneric, timeloop .eggs
> directories come from,
> but I did figure out that they don't get installed as eggs if you add them
> to the "test" extras in setup.py, e.g.:
>
> extras_require={
> 'docs': ['Sphinx>=1.5.2,<2.0'],
> 'test': REQUIRED_TEST_PACKAGES + INTERACTIVE_BEAM,
> 'gcp': GCP_REQUIREMENTS,
> 'interactive': INTERACTIVE_BEAM,
> },
>
> This is further proof of the wisdom of the pytest-runner deprecation
> notice <https://pypi.org/project/pytest-runner/> (emphasis mine):
> """
> Remove ‘pytest’ and any other testing requirements from ‘*tests_require*’,
> preferably removing the setup_requires option.
> """
>
> I believe we don't rely on the tests_require definition. Removing it might
> break developers running "python setup.py test", but the alternative is a
> simple "setup.py && pip install".
>
>
> On Tue, Nov 26, 2019 at 5:14 PM Chad Dombrova <ch...@gmail.com> wrote:
>
>> Sorry wrong link: https://github.com/apache/beam/pull/9915
>>
>>
>>
>> On Tue, Nov 26, 2019 at 5:12 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> I looked at #9959 but it doesn't seem to modify setup.py?
>>> The additional eggs for timeloop etc. are troubling though. Not sure
>>> where those come from.
>>>
>>> On Tue, Nov 26, 2019 at 4:59 PM Chad Dombrova <ch...@gmail.com> wrote:
>>>
>>>> Is setup_requires being used somewhere else, because I'm still getting
>>>> errors after removing it from sdks/python/setup.py.
>>>>
>>>> I removed it from this PR: https://github.com/apache/beam/pull/9959
>>>>
>>>> Here's the gradle scan:
>>>> https://scans.gradle.com/s/oinh5xpaly3dk/failure#top=0
>>>>
>>>> The error shows up differently than before when
>>>> setup_requries=['pytest-runner'] was present -- it's in a gradle traceback
>>>> now rather than the console log. I've also seen different packages listed
>>>> as the culprit (simplegeneric, timeloop).
>>>>
>>>> -chad
>>>>
>>>>
>>>>
>>>> On Tue, Nov 26, 2019 at 4:47 PM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>>> Chad, I believe the answer is the "setup_requires" line is causing the
>>>>> sdks/python/.eggs directory to be created.
>>>>>
>>>>> This command fails with the setup_requires line (same Errno 17), but
>>>>> succeeds without it:
>>>>> $ \rm -r .eggs/; ../../gradlew installGcpTest
>>>>> [~8 failed tasks]
>>>>> $ ls .eggs
>>>>> pytest_runner-5.2-py2.7.egg pytest_runner-5.2-py3.5.egg
>>>>> pytest_runner-5.2-py3.6.egg pytest_runner-5.2-py3.7.egg README.txt
>>>>>
>>>>> I'll go ahead and create a PR to remove setup_requires from setup.py.
>>>>>
>>>>> On Tue, Nov 26, 2019 at 4:16 PM Chad Dombrova <ch...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> It seems like the offending packages are those that only have source
>>>>>> distributions (i.e. no wheels). But why are the eggs being installed in
>>>>>> sdks/python/.eggs instead of into the virtualenv created by setupVirtualenv
>>>>>> gradle task or by tox?
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 26, 2019 at 3:59 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>
>>>>>>> Basically, I believe what's happening is that a new Gradle task was
>>>>>>> added that uses setup.py but doesn't have the same dependency on some main
>>>>>>> setup.py task that all others depend on (list sdist).
>>>>>>>
>>>>>>> On Tue, Nov 26, 2019 at 3:49 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>
>>>>>>>> Correction: the error is not gone after removing the line. I get
>>>>>>>> instead:
>>>>>>>> error: [Errno 17] File exists:
>>>>>>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/dill-0.3.1.1-py2.7.egg'
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Nov 26, 2019 at 3:45 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>>
>>>>>>>>> I managed to recreate one of the issues with this command:
>>>>>>>>> ~/src/beam/sdks/python$ \rm -r .eggs/ && for i in $(seq 2); do
>>>>>>>>> echo "python setup.py -q nosetests --tests
>>>>>>>>> apache_beam.pipeline_test:DoFnTest.test_incomparable_default &" | sh ; done
>>>>>>>>>
>>>>>>>>> This reliably gives me:
>>>>>>>>> OSError: [Errno 17] File exists:
>>>>>>>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/pytest_runner-5.2-py2.7.egg'
>>>>>>>>>
>>>>>>>>> If I remove this line from setup.py the error is gone:
>>>>>>>>> setup_requires=['pytest_runner'],
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Nov 26, 2019 at 2:54 PM Chad Dombrova <ch...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks for looking into this. It seems like it might be something
>>>>>>>>>> to do with data that is cached on the Jenkins slaves between runs, which
>>>>>>>>>> may be what prevents this from showing up locally?
>>>>>>>>>>
>>>>>>>>>> If your theory about setuptools is correct, and it sounds likely,
>>>>>>>>>> we should be able to lock down the version, which we should definitely be
>>>>>>>>>> doing for all of our dependencies.
>>>>>>>>>>
>>>>>>>>>> -chad
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Nov 26, 2019 at 1:33 PM Ahmet Altay <al...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I tried to debug but did not make much progress. I cannot
>>>>>>>>>>> reproduce locally, however all python precommits and postcommits are
>>>>>>>>>>> failing.
>>>>>>>>>>>
>>>>>>>>>>> One guess is, setuptools released a new version that does not
>>>>>>>>>>> support eggs a few days ago, that might be the cause (
>>>>>>>>>>> https://github.com/pypa/setuptools/blob/master/CHANGES.rst) but
>>>>>>>>>>> that should have reproduced locally.
>>>>>>>>>>> Maybe something is wrong with the jenkins machines, and we could
>>>>>>>>>>> perhaps bring them to a clean state.
>>>>>>>>>>>
>>>>>>>>>>> I suspected this being related to pytest somehow (as the first 4
>>>>>>>>>>> JIRAs had pytest in the error line) but the error Chad saw is different.
>>>>>>>>>>>
>>>>>>>>>>> +Valentyn Tymofieiev <va...@google.com> and +Yifan Zou
>>>>>>>>>>> <yi...@google.com> could you help with looking into this?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Ahmet
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Nov 26, 2019 at 9:14 AM Luke Cwik <lc...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I also started to see this on PRs that I'm reviewing.
>>>>>>>>>>>> BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
>>>>>>>>>>>> egg_info but this looks different then all of those so I filed BEAM-8831.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <
>>>>>>>>>>>> chadrik@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Actually, it looks like I'm getting the same error on multiple
>>>>>>>>>>>>> PRs: https://scans.gradle.com/s/ihfmrxr7evslw
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <
>>>>>>>>>>>>> chadrik@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>> The cython tests started failing on one of my PRs which were
>>>>>>>>>>>>>> succeeding before. The error is one that I've never seen before
>>>>>>>>>>>>>> (separated onto different lines to make it easier to read):
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Caused by: org.gradle.api.GradleException:
>>>>>>>>>>>>>> Could not copy file
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>>>>>>>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>>>>>>>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Followed immediately by an error about could not create a
>>>>>>>>>>>>>> directory of the same name. Here's the gradle scan:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Any ideas?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -chad
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
Re: cython test instability
Posted by Udi Meiri <eh...@google.com>.
I'm not sure about where the error with the simplegeneric, timeloop .eggs
directories come from,
but I did figure out that they don't get installed as eggs if you add them
to the "test" extras in setup.py, e.g.:
extras_require={
'docs': ['Sphinx>=1.5.2,<2.0'],
'test': REQUIRED_TEST_PACKAGES + INTERACTIVE_BEAM,
'gcp': GCP_REQUIREMENTS,
'interactive': INTERACTIVE_BEAM,
},
This is further proof of the wisdom of the pytest-runner deprecation notice
<https://pypi.org/project/pytest-runner/> (emphasis mine):
"""
Remove ‘pytest’ and any other testing requirements from ‘*tests_require*’,
preferably removing the setup_requires option.
"""
I believe we don't rely on the tests_require definition. Removing it might
break developers running "python setup.py test", but the alternative is a
simple "setup.py && pip install".
On Tue, Nov 26, 2019 at 5:14 PM Chad Dombrova <ch...@gmail.com> wrote:
> Sorry wrong link: https://github.com/apache/beam/pull/9915
>
>
>
> On Tue, Nov 26, 2019 at 5:12 PM Udi Meiri <eh...@google.com> wrote:
>
>> I looked at #9959 but it doesn't seem to modify setup.py?
>> The additional eggs for timeloop etc. are troubling though. Not sure
>> where those come from.
>>
>> On Tue, Nov 26, 2019 at 4:59 PM Chad Dombrova <ch...@gmail.com> wrote:
>>
>>> Is setup_requires being used somewhere else, because I'm still getting
>>> errors after removing it from sdks/python/setup.py.
>>>
>>> I removed it from this PR: https://github.com/apache/beam/pull/9959
>>>
>>> Here's the gradle scan:
>>> https://scans.gradle.com/s/oinh5xpaly3dk/failure#top=0
>>>
>>> The error shows up differently than before when
>>> setup_requries=['pytest-runner'] was present -- it's in a gradle traceback
>>> now rather than the console log. I've also seen different packages listed
>>> as the culprit (simplegeneric, timeloop).
>>>
>>> -chad
>>>
>>>
>>>
>>> On Tue, Nov 26, 2019 at 4:47 PM Udi Meiri <eh...@google.com> wrote:
>>>
>>>> Chad, I believe the answer is the "setup_requires" line is causing the
>>>> sdks/python/.eggs directory to be created.
>>>>
>>>> This command fails with the setup_requires line (same Errno 17), but
>>>> succeeds without it:
>>>> $ \rm -r .eggs/; ../../gradlew installGcpTest
>>>> [~8 failed tasks]
>>>> $ ls .eggs
>>>> pytest_runner-5.2-py2.7.egg pytest_runner-5.2-py3.5.egg
>>>> pytest_runner-5.2-py3.6.egg pytest_runner-5.2-py3.7.egg README.txt
>>>>
>>>> I'll go ahead and create a PR to remove setup_requires from setup.py.
>>>>
>>>> On Tue, Nov 26, 2019 at 4:16 PM Chad Dombrova <ch...@gmail.com>
>>>> wrote:
>>>>
>>>>> It seems like the offending packages are those that only have source
>>>>> distributions (i.e. no wheels). But why are the eggs being installed in
>>>>> sdks/python/.eggs instead of into the virtualenv created by setupVirtualenv
>>>>> gradle task or by tox?
>>>>>
>>>>>
>>>>> On Tue, Nov 26, 2019 at 3:59 PM Udi Meiri <eh...@google.com> wrote:
>>>>>
>>>>>> Basically, I believe what's happening is that a new Gradle task was
>>>>>> added that uses setup.py but doesn't have the same dependency on some main
>>>>>> setup.py task that all others depend on (list sdist).
>>>>>>
>>>>>> On Tue, Nov 26, 2019 at 3:49 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>
>>>>>>> Correction: the error is not gone after removing the line. I get
>>>>>>> instead:
>>>>>>> error: [Errno 17] File exists:
>>>>>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/dill-0.3.1.1-py2.7.egg'
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Nov 26, 2019 at 3:45 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>
>>>>>>>> I managed to recreate one of the issues with this command:
>>>>>>>> ~/src/beam/sdks/python$ \rm -r .eggs/ && for i in $(seq 2); do echo
>>>>>>>> "python setup.py -q nosetests --tests
>>>>>>>> apache_beam.pipeline_test:DoFnTest.test_incomparable_default &" | sh ; done
>>>>>>>>
>>>>>>>> This reliably gives me:
>>>>>>>> OSError: [Errno 17] File exists:
>>>>>>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/pytest_runner-5.2-py2.7.egg'
>>>>>>>>
>>>>>>>> If I remove this line from setup.py the error is gone:
>>>>>>>> setup_requires=['pytest_runner'],
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Nov 26, 2019 at 2:54 PM Chad Dombrova <ch...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks for looking into this. It seems like it might be something
>>>>>>>>> to do with data that is cached on the Jenkins slaves between runs, which
>>>>>>>>> may be what prevents this from showing up locally?
>>>>>>>>>
>>>>>>>>> If your theory about setuptools is correct, and it sounds likely,
>>>>>>>>> we should be able to lock down the version, which we should definitely be
>>>>>>>>> doing for all of our dependencies.
>>>>>>>>>
>>>>>>>>> -chad
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Nov 26, 2019 at 1:33 PM Ahmet Altay <al...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I tried to debug but did not make much progress. I cannot
>>>>>>>>>> reproduce locally, however all python precommits and postcommits are
>>>>>>>>>> failing.
>>>>>>>>>>
>>>>>>>>>> One guess is, setuptools released a new version that does not
>>>>>>>>>> support eggs a few days ago, that might be the cause (
>>>>>>>>>> https://github.com/pypa/setuptools/blob/master/CHANGES.rst) but
>>>>>>>>>> that should have reproduced locally.
>>>>>>>>>> Maybe something is wrong with the jenkins machines, and we could
>>>>>>>>>> perhaps bring them to a clean state.
>>>>>>>>>>
>>>>>>>>>> I suspected this being related to pytest somehow (as the first 4
>>>>>>>>>> JIRAs had pytest in the error line) but the error Chad saw is different.
>>>>>>>>>>
>>>>>>>>>> +Valentyn Tymofieiev <va...@google.com> and +Yifan Zou
>>>>>>>>>> <yi...@google.com> could you help with looking into this?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ahmet
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Nov 26, 2019 at 9:14 AM Luke Cwik <lc...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I also started to see this on PRs that I'm reviewing.
>>>>>>>>>>> BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
>>>>>>>>>>> egg_info but this looks different then all of those so I filed BEAM-8831.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <
>>>>>>>>>>> chadrik@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Actually, it looks like I'm getting the same error on multiple
>>>>>>>>>>>> PRs: https://scans.gradle.com/s/ihfmrxr7evslw
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <
>>>>>>>>>>>> chadrik@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>> The cython tests started failing on one of my PRs which were
>>>>>>>>>>>>> succeeding before. The error is one that I've never seen before
>>>>>>>>>>>>> (separated onto different lines to make it easier to read):
>>>>>>>>>>>>>
>>>>>>>>>>>>> Caused by: org.gradle.api.GradleException:
>>>>>>>>>>>>> Could not copy file
>>>>>>>>>>>>>
>>>>>>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>>>>>>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>>>>>>>>>>>>> to
>>>>>>>>>>>>>
>>>>>>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>>>>>>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Followed immediately by an error about could not create a
>>>>>>>>>>>>> directory of the same name. Here's the gradle scan:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any ideas?
>>>>>>>>>>>>>
>>>>>>>>>>>>> -chad
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
Re: cython test instability
Posted by Chad Dombrova <ch...@gmail.com>.
Sorry wrong link: https://github.com/apache/beam/pull/9915
On Tue, Nov 26, 2019 at 5:12 PM Udi Meiri <eh...@google.com> wrote:
> I looked at #9959 but it doesn't seem to modify setup.py?
> The additional eggs for timeloop etc. are troubling though. Not sure where
> those come from.
>
> On Tue, Nov 26, 2019 at 4:59 PM Chad Dombrova <ch...@gmail.com> wrote:
>
>> Is setup_requires being used somewhere else, because I'm still getting
>> errors after removing it from sdks/python/setup.py.
>>
>> I removed it from this PR: https://github.com/apache/beam/pull/9959
>>
>> Here's the gradle scan:
>> https://scans.gradle.com/s/oinh5xpaly3dk/failure#top=0
>>
>> The error shows up differently than before when
>> setup_requries=['pytest-runner'] was present -- it's in a gradle traceback
>> now rather than the console log. I've also seen different packages listed
>> as the culprit (simplegeneric, timeloop).
>>
>> -chad
>>
>>
>>
>> On Tue, Nov 26, 2019 at 4:47 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> Chad, I believe the answer is the "setup_requires" line is causing the
>>> sdks/python/.eggs directory to be created.
>>>
>>> This command fails with the setup_requires line (same Errno 17), but
>>> succeeds without it:
>>> $ \rm -r .eggs/; ../../gradlew installGcpTest
>>> [~8 failed tasks]
>>> $ ls .eggs
>>> pytest_runner-5.2-py2.7.egg pytest_runner-5.2-py3.5.egg
>>> pytest_runner-5.2-py3.6.egg pytest_runner-5.2-py3.7.egg README.txt
>>>
>>> I'll go ahead and create a PR to remove setup_requires from setup.py.
>>>
>>> On Tue, Nov 26, 2019 at 4:16 PM Chad Dombrova <ch...@gmail.com> wrote:
>>>
>>>> It seems like the offending packages are those that only have source
>>>> distributions (i.e. no wheels). But why are the eggs being installed in
>>>> sdks/python/.eggs instead of into the virtualenv created by setupVirtualenv
>>>> gradle task or by tox?
>>>>
>>>>
>>>> On Tue, Nov 26, 2019 at 3:59 PM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>>> Basically, I believe what's happening is that a new Gradle task was
>>>>> added that uses setup.py but doesn't have the same dependency on some main
>>>>> setup.py task that all others depend on (list sdist).
>>>>>
>>>>> On Tue, Nov 26, 2019 at 3:49 PM Udi Meiri <eh...@google.com> wrote:
>>>>>
>>>>>> Correction: the error is not gone after removing the line. I get
>>>>>> instead:
>>>>>> error: [Errno 17] File exists:
>>>>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/dill-0.3.1.1-py2.7.egg'
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 26, 2019 at 3:45 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>
>>>>>>> I managed to recreate one of the issues with this command:
>>>>>>> ~/src/beam/sdks/python$ \rm -r .eggs/ && for i in $(seq 2); do echo
>>>>>>> "python setup.py -q nosetests --tests
>>>>>>> apache_beam.pipeline_test:DoFnTest.test_incomparable_default &" | sh ; done
>>>>>>>
>>>>>>> This reliably gives me:
>>>>>>> OSError: [Errno 17] File exists:
>>>>>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/pytest_runner-5.2-py2.7.egg'
>>>>>>>
>>>>>>> If I remove this line from setup.py the error is gone:
>>>>>>> setup_requires=['pytest_runner'],
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Nov 26, 2019 at 2:54 PM Chad Dombrova <ch...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thanks for looking into this. It seems like it might be something
>>>>>>>> to do with data that is cached on the Jenkins slaves between runs, which
>>>>>>>> may be what prevents this from showing up locally?
>>>>>>>>
>>>>>>>> If your theory about setuptools is correct, and it sounds likely,
>>>>>>>> we should be able to lock down the version, which we should definitely be
>>>>>>>> doing for all of our dependencies.
>>>>>>>>
>>>>>>>> -chad
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Nov 26, 2019 at 1:33 PM Ahmet Altay <al...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I tried to debug but did not make much progress. I cannot
>>>>>>>>> reproduce locally, however all python precommits and postcommits are
>>>>>>>>> failing.
>>>>>>>>>
>>>>>>>>> One guess is, setuptools released a new version that does not
>>>>>>>>> support eggs a few days ago, that might be the cause (
>>>>>>>>> https://github.com/pypa/setuptools/blob/master/CHANGES.rst) but
>>>>>>>>> that should have reproduced locally.
>>>>>>>>> Maybe something is wrong with the jenkins machines, and we could
>>>>>>>>> perhaps bring them to a clean state.
>>>>>>>>>
>>>>>>>>> I suspected this being related to pytest somehow (as the first 4
>>>>>>>>> JIRAs had pytest in the error line) but the error Chad saw is different.
>>>>>>>>>
>>>>>>>>> +Valentyn Tymofieiev <va...@google.com> and +Yifan Zou
>>>>>>>>> <yi...@google.com> could you help with looking into this?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ahmet
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Nov 26, 2019 at 9:14 AM Luke Cwik <lc...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I also started to see this on PRs that I'm reviewing.
>>>>>>>>>> BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
>>>>>>>>>> egg_info but this looks different then all of those so I filed BEAM-8831.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <ch...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Actually, it looks like I'm getting the same error on multiple
>>>>>>>>>>> PRs: https://scans.gradle.com/s/ihfmrxr7evslw
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <
>>>>>>>>>>> chadrik@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>> The cython tests started failing on one of my PRs which were
>>>>>>>>>>>> succeeding before. The error is one that I've never seen before
>>>>>>>>>>>> (separated onto different lines to make it easier to read):
>>>>>>>>>>>>
>>>>>>>>>>>> Caused by: org.gradle.api.GradleException:
>>>>>>>>>>>> Could not copy file
>>>>>>>>>>>>
>>>>>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>>>>>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>>>>>>>>>>>> to
>>>>>>>>>>>>
>>>>>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>>>>>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>>>>>>>>>>>
>>>>>>>>>>>> Followed immediately by an error about could not create a
>>>>>>>>>>>> directory of the same name. Here's the gradle scan:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>>>>>>>>>>>
>>>>>>>>>>>> Any ideas?
>>>>>>>>>>>>
>>>>>>>>>>>> -chad
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
Re: cython test instability
Posted by Udi Meiri <eh...@google.com>.
I looked at #9959 but it doesn't seem to modify setup.py?
The additional eggs for timeloop etc. are troubling though. Not sure where
those come from.
On Tue, Nov 26, 2019 at 4:59 PM Chad Dombrova <ch...@gmail.com> wrote:
> Is setup_requires being used somewhere else, because I'm still getting
> errors after removing it from sdks/python/setup.py.
>
> I removed it from this PR: https://github.com/apache/beam/pull/9959
>
> Here's the gradle scan:
> https://scans.gradle.com/s/oinh5xpaly3dk/failure#top=0
>
> The error shows up differently than before when
> setup_requries=['pytest-runner'] was present -- it's in a gradle traceback
> now rather than the console log. I've also seen different packages listed
> as the culprit (simplegeneric, timeloop).
>
> -chad
>
>
>
> On Tue, Nov 26, 2019 at 4:47 PM Udi Meiri <eh...@google.com> wrote:
>
>> Chad, I believe the answer is the "setup_requires" line is causing the
>> sdks/python/.eggs directory to be created.
>>
>> This command fails with the setup_requires line (same Errno 17), but
>> succeeds without it:
>> $ \rm -r .eggs/; ../../gradlew installGcpTest
>> [~8 failed tasks]
>> $ ls .eggs
>> pytest_runner-5.2-py2.7.egg pytest_runner-5.2-py3.5.egg
>> pytest_runner-5.2-py3.6.egg pytest_runner-5.2-py3.7.egg README.txt
>>
>> I'll go ahead and create a PR to remove setup_requires from setup.py.
>>
>> On Tue, Nov 26, 2019 at 4:16 PM Chad Dombrova <ch...@gmail.com> wrote:
>>
>>> It seems like the offending packages are those that only have source
>>> distributions (i.e. no wheels). But why are the eggs being installed in
>>> sdks/python/.eggs instead of into the virtualenv created by setupVirtualenv
>>> gradle task or by tox?
>>>
>>>
>>> On Tue, Nov 26, 2019 at 3:59 PM Udi Meiri <eh...@google.com> wrote:
>>>
>>>> Basically, I believe what's happening is that a new Gradle task was
>>>> added that uses setup.py but doesn't have the same dependency on some main
>>>> setup.py task that all others depend on (list sdist).
>>>>
>>>> On Tue, Nov 26, 2019 at 3:49 PM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>>> Correction: the error is not gone after removing the line. I get
>>>>> instead:
>>>>> error: [Errno 17] File exists:
>>>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/dill-0.3.1.1-py2.7.egg'
>>>>>
>>>>>
>>>>> On Tue, Nov 26, 2019 at 3:45 PM Udi Meiri <eh...@google.com> wrote:
>>>>>
>>>>>> I managed to recreate one of the issues with this command:
>>>>>> ~/src/beam/sdks/python$ \rm -r .eggs/ && for i in $(seq 2); do echo
>>>>>> "python setup.py -q nosetests --tests
>>>>>> apache_beam.pipeline_test:DoFnTest.test_incomparable_default &" | sh ; done
>>>>>>
>>>>>> This reliably gives me:
>>>>>> OSError: [Errno 17] File exists:
>>>>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/pytest_runner-5.2-py2.7.egg'
>>>>>>
>>>>>> If I remove this line from setup.py the error is gone:
>>>>>> setup_requires=['pytest_runner'],
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 26, 2019 at 2:54 PM Chad Dombrova <ch...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks for looking into this. It seems like it might be something to
>>>>>>> do with data that is cached on the Jenkins slaves between runs, which may
>>>>>>> be what prevents this from showing up locally?
>>>>>>>
>>>>>>> If your theory about setuptools is correct, and it sounds likely, we
>>>>>>> should be able to lock down the version, which we should definitely be
>>>>>>> doing for all of our dependencies.
>>>>>>>
>>>>>>> -chad
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Nov 26, 2019 at 1:33 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I tried to debug but did not make much progress. I cannot reproduce
>>>>>>>> locally, however all python precommits and postcommits are failing.
>>>>>>>>
>>>>>>>> One guess is, setuptools released a new version that does not
>>>>>>>> support eggs a few days ago, that might be the cause (
>>>>>>>> https://github.com/pypa/setuptools/blob/master/CHANGES.rst) but
>>>>>>>> that should have reproduced locally.
>>>>>>>> Maybe something is wrong with the jenkins machines, and we could
>>>>>>>> perhaps bring them to a clean state.
>>>>>>>>
>>>>>>>> I suspected this being related to pytest somehow (as the first 4
>>>>>>>> JIRAs had pytest in the error line) but the error Chad saw is different.
>>>>>>>>
>>>>>>>> +Valentyn Tymofieiev <va...@google.com> and +Yifan Zou
>>>>>>>> <yi...@google.com> could you help with looking into this?
>>>>>>>>
>>>>>>>>
>>>>>>>> Ahmet
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Nov 26, 2019 at 9:14 AM Luke Cwik <lc...@google.com> wrote:
>>>>>>>>
>>>>>>>>> I also started to see this on PRs that I'm reviewing.
>>>>>>>>> BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
>>>>>>>>> egg_info but this looks different then all of those so I filed BEAM-8831.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <ch...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Actually, it looks like I'm getting the same error on multiple
>>>>>>>>>> PRs: https://scans.gradle.com/s/ihfmrxr7evslw
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <ch...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>> The cython tests started failing on one of my PRs which were
>>>>>>>>>>> succeeding before. The error is one that I've never seen before
>>>>>>>>>>> (separated onto different lines to make it easier to read):
>>>>>>>>>>>
>>>>>>>>>>> Caused by: org.gradle.api.GradleException:
>>>>>>>>>>> Could not copy file
>>>>>>>>>>>
>>>>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>>>>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>>>>>>>>>>> to
>>>>>>>>>>>
>>>>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>>>>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>>>>>>>>>>
>>>>>>>>>>> Followed immediately by an error about could not create a
>>>>>>>>>>> directory of the same name. Here's the gradle scan:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>>>>>>>>>>
>>>>>>>>>>> Any ideas?
>>>>>>>>>>>
>>>>>>>>>>> -chad
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
Re: cython test instability
Posted by Chad Dombrova <ch...@gmail.com>.
Is setup_requires being used somewhere else, because I'm still getting
errors after removing it from sdks/python/setup.py.
I removed it from this PR: https://github.com/apache/beam/pull/9959
Here's the gradle scan:
https://scans.gradle.com/s/oinh5xpaly3dk/failure#top=0
The error shows up differently than before when
setup_requries=['pytest-runner'] was present -- it's in a gradle traceback
now rather than the console log. I've also seen different packages listed
as the culprit (simplegeneric, timeloop).
-chad
On Tue, Nov 26, 2019 at 4:47 PM Udi Meiri <eh...@google.com> wrote:
> Chad, I believe the answer is the "setup_requires" line is causing the
> sdks/python/.eggs directory to be created.
>
> This command fails with the setup_requires line (same Errno 17), but
> succeeds without it:
> $ \rm -r .eggs/; ../../gradlew installGcpTest
> [~8 failed tasks]
> $ ls .eggs
> pytest_runner-5.2-py2.7.egg pytest_runner-5.2-py3.5.egg
> pytest_runner-5.2-py3.6.egg pytest_runner-5.2-py3.7.egg README.txt
>
> I'll go ahead and create a PR to remove setup_requires from setup.py.
>
> On Tue, Nov 26, 2019 at 4:16 PM Chad Dombrova <ch...@gmail.com> wrote:
>
>> It seems like the offending packages are those that only have source
>> distributions (i.e. no wheels). But why are the eggs being installed in
>> sdks/python/.eggs instead of into the virtualenv created by setupVirtualenv
>> gradle task or by tox?
>>
>>
>> On Tue, Nov 26, 2019 at 3:59 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> Basically, I believe what's happening is that a new Gradle task was
>>> added that uses setup.py but doesn't have the same dependency on some main
>>> setup.py task that all others depend on (list sdist).
>>>
>>> On Tue, Nov 26, 2019 at 3:49 PM Udi Meiri <eh...@google.com> wrote:
>>>
>>>> Correction: the error is not gone after removing the line. I get
>>>> instead:
>>>> error: [Errno 17] File exists:
>>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/dill-0.3.1.1-py2.7.egg'
>>>>
>>>>
>>>> On Tue, Nov 26, 2019 at 3:45 PM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>>> I managed to recreate one of the issues with this command:
>>>>> ~/src/beam/sdks/python$ \rm -r .eggs/ && for i in $(seq 2); do echo
>>>>> "python setup.py -q nosetests --tests
>>>>> apache_beam.pipeline_test:DoFnTest.test_incomparable_default &" | sh ; done
>>>>>
>>>>> This reliably gives me:
>>>>> OSError: [Errno 17] File exists:
>>>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/pytest_runner-5.2-py2.7.egg'
>>>>>
>>>>> If I remove this line from setup.py the error is gone:
>>>>> setup_requires=['pytest_runner'],
>>>>>
>>>>>
>>>>> On Tue, Nov 26, 2019 at 2:54 PM Chad Dombrova <ch...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks for looking into this. It seems like it might be something to
>>>>>> do with data that is cached on the Jenkins slaves between runs, which may
>>>>>> be what prevents this from showing up locally?
>>>>>>
>>>>>> If your theory about setuptools is correct, and it sounds likely, we
>>>>>> should be able to lock down the version, which we should definitely be
>>>>>> doing for all of our dependencies.
>>>>>>
>>>>>> -chad
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 26, 2019 at 1:33 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>> I tried to debug but did not make much progress. I cannot reproduce
>>>>>>> locally, however all python precommits and postcommits are failing.
>>>>>>>
>>>>>>> One guess is, setuptools released a new version that does not
>>>>>>> support eggs a few days ago, that might be the cause (
>>>>>>> https://github.com/pypa/setuptools/blob/master/CHANGES.rst) but
>>>>>>> that should have reproduced locally.
>>>>>>> Maybe something is wrong with the jenkins machines, and we could
>>>>>>> perhaps bring them to a clean state.
>>>>>>>
>>>>>>> I suspected this being related to pytest somehow (as the first 4
>>>>>>> JIRAs had pytest in the error line) but the error Chad saw is different.
>>>>>>>
>>>>>>> +Valentyn Tymofieiev <va...@google.com> and +Yifan Zou
>>>>>>> <yi...@google.com> could you help with looking into this?
>>>>>>>
>>>>>>>
>>>>>>> Ahmet
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Nov 26, 2019 at 9:14 AM Luke Cwik <lc...@google.com> wrote:
>>>>>>>
>>>>>>>> I also started to see this on PRs that I'm reviewing.
>>>>>>>> BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
>>>>>>>> egg_info but this looks different then all of those so I filed BEAM-8831.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <ch...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Actually, it looks like I'm getting the same error on multiple
>>>>>>>>> PRs: https://scans.gradle.com/s/ihfmrxr7evslw
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <ch...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>> The cython tests started failing on one of my PRs which were
>>>>>>>>>> succeeding before. The error is one that I've never seen before
>>>>>>>>>> (separated onto different lines to make it easier to read):
>>>>>>>>>>
>>>>>>>>>> Caused by: org.gradle.api.GradleException:
>>>>>>>>>> Could not copy file
>>>>>>>>>>
>>>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>>>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>>>>>>>>>> to
>>>>>>>>>>
>>>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>>>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>>>>>>>>>
>>>>>>>>>> Followed immediately by an error about could not create a
>>>>>>>>>> directory of the same name. Here's the gradle scan:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>>>>>>>>>
>>>>>>>>>> Any ideas?
>>>>>>>>>>
>>>>>>>>>> -chad
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
Re: cython test instability
Posted by Udi Meiri <eh...@google.com>.
Chad, I believe the answer is the "setup_requires" line is causing the
sdks/python/.eggs directory to be created.
This command fails with the setup_requires line (same Errno 17), but
succeeds without it:
$ \rm -r .eggs/; ../../gradlew installGcpTest
[~8 failed tasks]
$ ls .eggs
pytest_runner-5.2-py2.7.egg pytest_runner-5.2-py3.5.egg
pytest_runner-5.2-py3.6.egg pytest_runner-5.2-py3.7.egg README.txt
I'll go ahead and create a PR to remove setup_requires from setup.py.
On Tue, Nov 26, 2019 at 4:16 PM Chad Dombrova <ch...@gmail.com> wrote:
> It seems like the offending packages are those that only have source
> distributions (i.e. no wheels). But why are the eggs being installed in
> sdks/python/.eggs instead of into the virtualenv created by setupVirtualenv
> gradle task or by tox?
>
>
> On Tue, Nov 26, 2019 at 3:59 PM Udi Meiri <eh...@google.com> wrote:
>
>> Basically, I believe what's happening is that a new Gradle task was added
>> that uses setup.py but doesn't have the same dependency on some main
>> setup.py task that all others depend on (list sdist).
>>
>> On Tue, Nov 26, 2019 at 3:49 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> Correction: the error is not gone after removing the line. I get instead:
>>> error: [Errno 17] File exists:
>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/dill-0.3.1.1-py2.7.egg'
>>>
>>>
>>> On Tue, Nov 26, 2019 at 3:45 PM Udi Meiri <eh...@google.com> wrote:
>>>
>>>> I managed to recreate one of the issues with this command:
>>>> ~/src/beam/sdks/python$ \rm -r .eggs/ && for i in $(seq 2); do echo
>>>> "python setup.py -q nosetests --tests
>>>> apache_beam.pipeline_test:DoFnTest.test_incomparable_default &" | sh ; done
>>>>
>>>> This reliably gives me:
>>>> OSError: [Errno 17] File exists:
>>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/pytest_runner-5.2-py2.7.egg'
>>>>
>>>> If I remove this line from setup.py the error is gone:
>>>> setup_requires=['pytest_runner'],
>>>>
>>>>
>>>> On Tue, Nov 26, 2019 at 2:54 PM Chad Dombrova <ch...@gmail.com>
>>>> wrote:
>>>>
>>>>> Thanks for looking into this. It seems like it might be something to
>>>>> do with data that is cached on the Jenkins slaves between runs, which may
>>>>> be what prevents this from showing up locally?
>>>>>
>>>>> If your theory about setuptools is correct, and it sounds likely, we
>>>>> should be able to lock down the version, which we should definitely be
>>>>> doing for all of our dependencies.
>>>>>
>>>>> -chad
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Nov 26, 2019 at 1:33 PM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>> I tried to debug but did not make much progress. I cannot reproduce
>>>>>> locally, however all python precommits and postcommits are failing.
>>>>>>
>>>>>> One guess is, setuptools released a new version that does not support
>>>>>> eggs a few days ago, that might be the cause (
>>>>>> https://github.com/pypa/setuptools/blob/master/CHANGES.rst) but that
>>>>>> should have reproduced locally.
>>>>>> Maybe something is wrong with the jenkins machines, and we could
>>>>>> perhaps bring them to a clean state.
>>>>>>
>>>>>> I suspected this being related to pytest somehow (as the first 4
>>>>>> JIRAs had pytest in the error line) but the error Chad saw is different.
>>>>>>
>>>>>> +Valentyn Tymofieiev <va...@google.com> and +Yifan Zou
>>>>>> <yi...@google.com> could you help with looking into this?
>>>>>>
>>>>>>
>>>>>> Ahmet
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 26, 2019 at 9:14 AM Luke Cwik <lc...@google.com> wrote:
>>>>>>
>>>>>>> I also started to see this on PRs that I'm reviewing.
>>>>>>> BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
>>>>>>> egg_info but this looks different then all of those so I filed BEAM-8831.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <ch...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Actually, it looks like I'm getting the same error on multiple PRs:
>>>>>>>> https://scans.gradle.com/s/ihfmrxr7evslw
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <ch...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>> The cython tests started failing on one of my PRs which were
>>>>>>>>> succeeding before. The error is one that I've never seen before
>>>>>>>>> (separated onto different lines to make it easier to read):
>>>>>>>>>
>>>>>>>>> Caused by: org.gradle.api.GradleException:
>>>>>>>>> Could not copy file
>>>>>>>>>
>>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>>>>>>>>> to
>>>>>>>>>
>>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>>>>>>>>
>>>>>>>>> Followed immediately by an error about could not create a
>>>>>>>>> directory of the same name. Here's the gradle scan:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>>>>>>>>
>>>>>>>>> Any ideas?
>>>>>>>>>
>>>>>>>>> -chad
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
Re: cython test instability
Posted by Chad Dombrova <ch...@gmail.com>.
It seems like the offending packages are those that only have source
distributions (i.e. no wheels). But why are the eggs being installed in
sdks/python/.eggs instead of into the virtualenv created by setupVirtualenv
gradle task or by tox?
On Tue, Nov 26, 2019 at 3:59 PM Udi Meiri <eh...@google.com> wrote:
> Basically, I believe what's happening is that a new Gradle task was added
> that uses setup.py but doesn't have the same dependency on some main
> setup.py task that all others depend on (list sdist).
>
> On Tue, Nov 26, 2019 at 3:49 PM Udi Meiri <eh...@google.com> wrote:
>
>> Correction: the error is not gone after removing the line. I get instead:
>> error: [Errno 17] File exists:
>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/dill-0.3.1.1-py2.7.egg'
>>
>>
>> On Tue, Nov 26, 2019 at 3:45 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> I managed to recreate one of the issues with this command:
>>> ~/src/beam/sdks/python$ \rm -r .eggs/ && for i in $(seq 2); do echo
>>> "python setup.py -q nosetests --tests
>>> apache_beam.pipeline_test:DoFnTest.test_incomparable_default &" | sh ; done
>>>
>>> This reliably gives me:
>>> OSError: [Errno 17] File exists:
>>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/pytest_runner-5.2-py2.7.egg'
>>>
>>> If I remove this line from setup.py the error is gone:
>>> setup_requires=['pytest_runner'],
>>>
>>>
>>> On Tue, Nov 26, 2019 at 2:54 PM Chad Dombrova <ch...@gmail.com> wrote:
>>>
>>>> Thanks for looking into this. It seems like it might be something to do
>>>> with data that is cached on the Jenkins slaves between runs, which may be
>>>> what prevents this from showing up locally?
>>>>
>>>> If your theory about setuptools is correct, and it sounds likely, we
>>>> should be able to lock down the version, which we should definitely be
>>>> doing for all of our dependencies.
>>>>
>>>> -chad
>>>>
>>>>
>>>>
>>>> On Tue, Nov 26, 2019 at 1:33 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>> I tried to debug but did not make much progress. I cannot reproduce
>>>>> locally, however all python precommits and postcommits are failing.
>>>>>
>>>>> One guess is, setuptools released a new version that does not support
>>>>> eggs a few days ago, that might be the cause (
>>>>> https://github.com/pypa/setuptools/blob/master/CHANGES.rst) but that
>>>>> should have reproduced locally.
>>>>> Maybe something is wrong with the jenkins machines, and we could
>>>>> perhaps bring them to a clean state.
>>>>>
>>>>> I suspected this being related to pytest somehow (as the first 4 JIRAs
>>>>> had pytest in the error line) but the error Chad saw is different.
>>>>>
>>>>> +Valentyn Tymofieiev <va...@google.com> and +Yifan Zou
>>>>> <yi...@google.com> could you help with looking into this?
>>>>>
>>>>>
>>>>> Ahmet
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Nov 26, 2019 at 9:14 AM Luke Cwik <lc...@google.com> wrote:
>>>>>
>>>>>> I also started to see this on PRs that I'm reviewing.
>>>>>> BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
>>>>>> egg_info but this looks different then all of those so I filed BEAM-8831.
>>>>>>
>>>>>>
>>>>>> On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <ch...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Actually, it looks like I'm getting the same error on multiple PRs:
>>>>>>> https://scans.gradle.com/s/ihfmrxr7evslw
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <ch...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>> The cython tests started failing on one of my PRs which were
>>>>>>>> succeeding before. The error is one that I've never seen before
>>>>>>>> (separated onto different lines to make it easier to read):
>>>>>>>>
>>>>>>>> Caused by: org.gradle.api.GradleException:
>>>>>>>> Could not copy file
>>>>>>>>
>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>>>>>>>> to
>>>>>>>>
>>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>>>>>>>
>>>>>>>> Followed immediately by an error about could not create a directory
>>>>>>>> of the same name. Here's the gradle scan:
>>>>>>>>
>>>>>>>>
>>>>>>>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>>>>>>>
>>>>>>>> Any ideas?
>>>>>>>>
>>>>>>>> -chad
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
Re: cython test instability
Posted by Udi Meiri <eh...@google.com>.
Basically, I believe what's happening is that a new Gradle task was added
that uses setup.py but doesn't have the same dependency on some main
setup.py task that all others depend on (list sdist).
On Tue, Nov 26, 2019 at 3:49 PM Udi Meiri <eh...@google.com> wrote:
> Correction: the error is not gone after removing the line. I get instead:
> error: [Errno 17] File exists:
> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/dill-0.3.1.1-py2.7.egg'
>
>
> On Tue, Nov 26, 2019 at 3:45 PM Udi Meiri <eh...@google.com> wrote:
>
>> I managed to recreate one of the issues with this command:
>> ~/src/beam/sdks/python$ \rm -r .eggs/ && for i in $(seq 2); do echo
>> "python setup.py -q nosetests --tests
>> apache_beam.pipeline_test:DoFnTest.test_incomparable_default &" | sh ; done
>>
>> This reliably gives me:
>> OSError: [Errno 17] File exists:
>> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/pytest_runner-5.2-py2.7.egg'
>>
>> If I remove this line from setup.py the error is gone:
>> setup_requires=['pytest_runner'],
>>
>>
>> On Tue, Nov 26, 2019 at 2:54 PM Chad Dombrova <ch...@gmail.com> wrote:
>>
>>> Thanks for looking into this. It seems like it might be something to do
>>> with data that is cached on the Jenkins slaves between runs, which may be
>>> what prevents this from showing up locally?
>>>
>>> If your theory about setuptools is correct, and it sounds likely, we
>>> should be able to lock down the version, which we should definitely be
>>> doing for all of our dependencies.
>>>
>>> -chad
>>>
>>>
>>>
>>> On Tue, Nov 26, 2019 at 1:33 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>> I tried to debug but did not make much progress. I cannot reproduce
>>>> locally, however all python precommits and postcommits are failing.
>>>>
>>>> One guess is, setuptools released a new version that does not support
>>>> eggs a few days ago, that might be the cause (
>>>> https://github.com/pypa/setuptools/blob/master/CHANGES.rst) but that
>>>> should have reproduced locally.
>>>> Maybe something is wrong with the jenkins machines, and we could
>>>> perhaps bring them to a clean state.
>>>>
>>>> I suspected this being related to pytest somehow (as the first 4 JIRAs
>>>> had pytest in the error line) but the error Chad saw is different.
>>>>
>>>> +Valentyn Tymofieiev <va...@google.com> and +Yifan Zou
>>>> <yi...@google.com> could you help with looking into this?
>>>>
>>>>
>>>> Ahmet
>>>>
>>>>
>>>>
>>>> On Tue, Nov 26, 2019 at 9:14 AM Luke Cwik <lc...@google.com> wrote:
>>>>
>>>>> I also started to see this on PRs that I'm reviewing.
>>>>> BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
>>>>> egg_info but this looks different then all of those so I filed BEAM-8831.
>>>>>
>>>>>
>>>>> On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <ch...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Actually, it looks like I'm getting the same error on multiple PRs:
>>>>>> https://scans.gradle.com/s/ihfmrxr7evslw
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <ch...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>> The cython tests started failing on one of my PRs which were
>>>>>>> succeeding before. The error is one that I've never seen before
>>>>>>> (separated onto different lines to make it easier to read):
>>>>>>>
>>>>>>> Caused by: org.gradle.api.GradleException:
>>>>>>> Could not copy file
>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>>>>>>> to
>>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>>>>>>
>>>>>>> Followed immediately by an error about could not create a directory
>>>>>>> of the same name. Here's the gradle scan:
>>>>>>>
>>>>>>>
>>>>>>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>>>>>>
>>>>>>> Any ideas?
>>>>>>>
>>>>>>> -chad
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
Re: cython test instability
Posted by Udi Meiri <eh...@google.com>.
Correction: the error is not gone after removing the line. I get instead:
error: [Errno 17] File exists:
'/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/dill-0.3.1.1-py2.7.egg'
On Tue, Nov 26, 2019 at 3:45 PM Udi Meiri <eh...@google.com> wrote:
> I managed to recreate one of the issues with this command:
> ~/src/beam/sdks/python$ \rm -r .eggs/ && for i in $(seq 2); do echo
> "python setup.py -q nosetests --tests
> apache_beam.pipeline_test:DoFnTest.test_incomparable_default &" | sh ; done
>
> This reliably gives me:
> OSError: [Errno 17] File exists:
> '/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/pytest_runner-5.2-py2.7.egg'
>
> If I remove this line from setup.py the error is gone:
> setup_requires=['pytest_runner'],
>
>
> On Tue, Nov 26, 2019 at 2:54 PM Chad Dombrova <ch...@gmail.com> wrote:
>
>> Thanks for looking into this. It seems like it might be something to do
>> with data that is cached on the Jenkins slaves between runs, which may be
>> what prevents this from showing up locally?
>>
>> If your theory about setuptools is correct, and it sounds likely, we
>> should be able to lock down the version, which we should definitely be
>> doing for all of our dependencies.
>>
>> -chad
>>
>>
>>
>> On Tue, Nov 26, 2019 at 1:33 PM Ahmet Altay <al...@google.com> wrote:
>>
>>> I tried to debug but did not make much progress. I cannot reproduce
>>> locally, however all python precommits and postcommits are failing.
>>>
>>> One guess is, setuptools released a new version that does not support
>>> eggs a few days ago, that might be the cause (
>>> https://github.com/pypa/setuptools/blob/master/CHANGES.rst) but that
>>> should have reproduced locally.
>>> Maybe something is wrong with the jenkins machines, and we could perhaps
>>> bring them to a clean state.
>>>
>>> I suspected this being related to pytest somehow (as the first 4 JIRAs
>>> had pytest in the error line) but the error Chad saw is different.
>>>
>>> +Valentyn Tymofieiev <va...@google.com> and +Yifan Zou
>>> <yi...@google.com> could you help with looking into this?
>>>
>>>
>>> Ahmet
>>>
>>>
>>>
>>> On Tue, Nov 26, 2019 at 9:14 AM Luke Cwik <lc...@google.com> wrote:
>>>
>>>> I also started to see this on PRs that I'm reviewing.
>>>> BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
>>>> egg_info but this looks different then all of those so I filed BEAM-8831.
>>>>
>>>>
>>>> On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <ch...@gmail.com>
>>>> wrote:
>>>>
>>>>> Actually, it looks like I'm getting the same error on multiple PRs:
>>>>> https://scans.gradle.com/s/ihfmrxr7evslw
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <ch...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>> The cython tests started failing on one of my PRs which were
>>>>>> succeeding before. The error is one that I've never seen before
>>>>>> (separated onto different lines to make it easier to read):
>>>>>>
>>>>>> Caused by: org.gradle.api.GradleException:
>>>>>> Could not copy file
>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>>>>>> to
>>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>>>>>
>>>>>> Followed immediately by an error about could not create a directory
>>>>>> of the same name. Here's the gradle scan:
>>>>>>
>>>>>>
>>>>>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>>>>>
>>>>>> Any ideas?
>>>>>>
>>>>>> -chad
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
Re: cython test instability
Posted by Udi Meiri <eh...@google.com>.
I managed to recreate one of the issues with this command:
~/src/beam/sdks/python$ \rm -r .eggs/ && for i in $(seq 2); do echo "python
setup.py -q nosetests --tests
apache_beam.pipeline_test:DoFnTest.test_incomparable_default &" | sh ; done
This reliably gives me:
OSError: [Errno 17] File exists:
'/usr/local/google/home/ehudm/src/beam/sdks/python/.eggs/pytest_runner-5.2-py2.7.egg'
If I remove this line from setup.py the error is gone:
setup_requires=['pytest_runner'],
On Tue, Nov 26, 2019 at 2:54 PM Chad Dombrova <ch...@gmail.com> wrote:
> Thanks for looking into this. It seems like it might be something to do
> with data that is cached on the Jenkins slaves between runs, which may be
> what prevents this from showing up locally?
>
> If your theory about setuptools is correct, and it sounds likely, we
> should be able to lock down the version, which we should definitely be
> doing for all of our dependencies.
>
> -chad
>
>
>
> On Tue, Nov 26, 2019 at 1:33 PM Ahmet Altay <al...@google.com> wrote:
>
>> I tried to debug but did not make much progress. I cannot reproduce
>> locally, however all python precommits and postcommits are failing.
>>
>> One guess is, setuptools released a new version that does not support
>> eggs a few days ago, that might be the cause (
>> https://github.com/pypa/setuptools/blob/master/CHANGES.rst) but that
>> should have reproduced locally.
>> Maybe something is wrong with the jenkins machines, and we could perhaps
>> bring them to a clean state.
>>
>> I suspected this being related to pytest somehow (as the first 4 JIRAs
>> had pytest in the error line) but the error Chad saw is different.
>>
>> +Valentyn Tymofieiev <va...@google.com> and +Yifan Zou
>> <yi...@google.com> could you help with looking into this?
>>
>>
>> Ahmet
>>
>>
>>
>> On Tue, Nov 26, 2019 at 9:14 AM Luke Cwik <lc...@google.com> wrote:
>>
>>> I also started to see this on PRs that I'm reviewing.
>>> BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
>>> egg_info but this looks different then all of those so I filed BEAM-8831.
>>>
>>>
>>> On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <ch...@gmail.com>
>>> wrote:
>>>
>>>> Actually, it looks like I'm getting the same error on multiple PRs:
>>>> https://scans.gradle.com/s/ihfmrxr7evslw
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <ch...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>> The cython tests started failing on one of my PRs which were
>>>>> succeeding before. The error is one that I've never seen before
>>>>> (separated onto different lines to make it easier to read):
>>>>>
>>>>> Caused by: org.gradle.api.GradleException:
>>>>> Could not copy file
>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>>>>> to
>>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>>>>
>>>>> Followed immediately by an error about could not create a directory of
>>>>> the same name. Here's the gradle scan:
>>>>>
>>>>>
>>>>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>>>>
>>>>> Any ideas?
>>>>>
>>>>> -chad
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
Re: cython test instability
Posted by Chad Dombrova <ch...@gmail.com>.
Thanks for looking into this. It seems like it might be something to do
with data that is cached on the Jenkins slaves between runs, which may be
what prevents this from showing up locally?
If your theory about setuptools is correct, and it sounds likely, we should
be able to lock down the version, which we should definitely be doing for
all of our dependencies.
-chad
On Tue, Nov 26, 2019 at 1:33 PM Ahmet Altay <al...@google.com> wrote:
> I tried to debug but did not make much progress. I cannot reproduce
> locally, however all python precommits and postcommits are failing.
>
> One guess is, setuptools released a new version that does not support eggs
> a few days ago, that might be the cause (
> https://github.com/pypa/setuptools/blob/master/CHANGES.rst) but that
> should have reproduced locally.
> Maybe something is wrong with the jenkins machines, and we could perhaps
> bring them to a clean state.
>
> I suspected this being related to pytest somehow (as the first 4 JIRAs had
> pytest in the error line) but the error Chad saw is different.
>
> +Valentyn Tymofieiev <va...@google.com> and +Yifan Zou
> <yi...@google.com> could you help with looking into this?
>
>
> Ahmet
>
>
>
> On Tue, Nov 26, 2019 at 9:14 AM Luke Cwik <lc...@google.com> wrote:
>
>> I also started to see this on PRs that I'm reviewing.
>> BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
>> egg_info but this looks different then all of those so I filed BEAM-8831.
>>
>>
>> On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <ch...@gmail.com> wrote:
>>
>>> Actually, it looks like I'm getting the same error on multiple PRs:
>>> https://scans.gradle.com/s/ihfmrxr7evslw
>>>
>>>
>>>
>>>
>>> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <ch...@gmail.com>
>>> wrote:
>>>
>>>> Hi all,
>>>> The cython tests started failing on one of my PRs which were succeeding
>>>> before. The error is one that I've never seen before (separated onto
>>>> different lines to make it easier to read):
>>>>
>>>> Caused by: org.gradle.api.GradleException:
>>>> Could not copy file
>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>>>> to
>>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>>>
>>>> Followed immediately by an error about could not create a directory of
>>>> the same name. Here's the gradle scan:
>>>>
>>>>
>>>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>>>
>>>> Any ideas?
>>>>
>>>> -chad
>>>>
>>>>
>>>>
>>>>
>>>>
Re: cython test instability
Posted by Ahmet Altay <al...@google.com>.
I tried to debug but did not make much progress. I cannot reproduce
locally, however all python precommits and postcommits are failing.
One guess is, setuptools released a new version that does not support eggs
a few days ago, that might be the cause (
https://github.com/pypa/setuptools/blob/master/CHANGES.rst) but that should
have reproduced locally.
Maybe something is wrong with the jenkins machines, and we could perhaps
bring them to a clean state.
I suspected this being related to pytest somehow (as the first 4 JIRAs had
pytest in the error line) but the error Chad saw is different.
+Valentyn Tymofieiev <va...@google.com> and +Yifan Zou
<yi...@google.com> could you help with looking into this?
Ahmet
On Tue, Nov 26, 2019 at 9:14 AM Luke Cwik <lc...@google.com> wrote:
> I also started to see this on PRs that I'm reviewing.
> BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
> egg_info but this looks different then all of those so I filed BEAM-8831.
>
>
> On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <ch...@gmail.com> wrote:
>
>> Actually, it looks like I'm getting the same error on multiple PRs:
>> https://scans.gradle.com/s/ihfmrxr7evslw
>>
>>
>>
>>
>> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <ch...@gmail.com> wrote:
>>
>>> Hi all,
>>> The cython tests started failing on one of my PRs which were succeeding
>>> before. The error is one that I've never seen before (separated onto
>>> different lines to make it easier to read):
>>>
>>> Caused by: org.gradle.api.GradleException:
>>> Could not copy file
>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>>> to
>>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>>
>>> Followed immediately by an error about could not create a directory of
>>> the same name. Here's the gradle scan:
>>>
>>>
>>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>>
>>> Any ideas?
>>>
>>> -chad
>>>
>>>
>>>
>>>
>>>
Re: cython test instability
Posted by Luke Cwik <lc...@google.com>.
I also started to see this on PRs that I'm reviewing.
BEAM-8793, BEAM-8653, BEAM-8631, BEAM-8249 mention issues with setup.py and
egg_info but this looks different then all of those so I filed BEAM-8831.
On Mon, Nov 25, 2019 at 10:27 PM Chad Dombrova <ch...@gmail.com> wrote:
> Actually, it looks like I'm getting the same error on multiple PRs:
> https://scans.gradle.com/s/ihfmrxr7evslw
>
>
>
>
> On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <ch...@gmail.com> wrote:
>
>> Hi all,
>> The cython tests started failing on one of my PRs which were succeeding
>> before. The error is one that I've never seen before (separated onto
>> different lines to make it easier to read):
>>
>> Caused by: org.gradle.api.GradleException:
>> Could not copy file
>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
>> to
>> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
>> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>>
>> Followed immediately by an error about could not create a directory of
>> the same name. Here's the gradle scan:
>>
>>
>> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>>
>> Any ideas?
>>
>> -chad
>>
>>
>>
>>
>>
Re: cython test instability
Posted by Chad Dombrova <ch...@gmail.com>.
Actually, it looks like I'm getting the same error on multiple PRs:
https://scans.gradle.com/s/ihfmrxr7evslw
On Mon, Nov 25, 2019 at 10:26 PM Chad Dombrova <ch...@gmail.com> wrote:
> Hi all,
> The cython tests started failing on one of my PRs which were succeeding
> before. The error is one that I've never seen before (separated onto
> different lines to make it easier to read):
>
> Caused by: org.gradle.api.GradleException:
> Could not copy file
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
> /src/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'
> to
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit@2
> /src/sdks/python/test-suites/tox/py2/build/srcs/sdks/python/.eggs/simplegeneric-0.8.1-py2.7.egg'.
>
> Followed immediately by an error about could not create a directory of the
> same name. Here's the gradle scan:
>
>
> https://scans.gradle.com/s/ihfmrxr7evslw/failure?openFailures=WzFd&openStackTraces=WzZd#top=0
>
> Any ideas?
>
> -chad
>
>
>
>
>