[jira] [Commented] (BEAM-1924) pip download failed in post commit

["Sourabh Bajaj (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963254#comment-15963254 ] 

Sourabh Bajaj commented on BEAM-1924:
-------------------------------------

One improvement that comes to mind is that we should pass the subprocess.PIPE options in this call similar to how we do it in the juliaset example

> pip download failed in post commit
> ----------------------------------
>
>                 Key: BEAM-1924
>                 URL: https://issues.apache.org/jira/browse/BEAM-1924
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-py
>            Reporter: Ahmet Altay
>
> A post commit failed with a pip failure:
> https://builds.apache.org/job/beam_PostCommit_Python_Verify/1801/consoleFull
> Captured output is not clear enough to tell what went wrong:
> ======================================================================
> ERROR: test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/transforms/ptransform_test.py", line 207, in test_par_do_with_multiple_outputs_and_using_return
>     pipeline.run()
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/test_pipeline.py", line 91, in run
>     result = super(TestPipeline, self).run()
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/pipeline.py", line 160, in run
>     self.to_runner_api(), self.runner, self.options).run(False)
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/pipeline.py", line 169, in run
>     return self.runner.run(self)
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py", line 39, in run
>     self.result = super(TestDataflowRunner, self).run(pipeline)
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py", line 175, in run
>     self.dataflow_client.create_job(self.job), self)
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/utils/retry.py", line 174, in wrapper
>     return fun(*args, **kwargs)
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", line 425, in create_job
>     self.create_job_description(job)
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", line 446, in create_job_description
>     job.options, file_copy=self._gcs_file_copy)
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/dependency.py", line 306, in stage_job_resources
>     setup_options.requirements_file, requirements_cache_path)
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/runners/dataflow/internal/dependency.py", line 242, in _populate_requirements_cache
>     processes.check_call(cmd_args)
>   File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/apache_beam/utils/processes.py", line 40, in check_call
>     return subprocess.check_call(*args, **kwargs)
>   File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
>     raise CalledProcessError(retcode, cmd)
> CalledProcessError: Command '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/sdks/python/bin/python', '-m', 'pip', 'install', '--download', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--no-binary', ':all:']' returned non-zero exit status 2
> Any ideas on how we can debug this failure or improve for the future? (cc:
>  [~markflyhigh] [~sb2nov])



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)