You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Beam JIRA Bot (Jira)" <ji...@apache.org> on 2021/10/19 17:25:01 UTC
[jira] [Commented] (BEAM-12779) apache beam is not able to find its
library when running via DataflowRunner
[ https://issues.apache.org/jira/browse/BEAM-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430651#comment-17430651 ]
Beam JIRA Bot commented on BEAM-12779:
--------------------------------------
This issue is P2 but has been unassigned without any comment for 60 days so it has been labeled "stale-P2". If this issue is still affecting you, we care! Please comment and remove the label. Otherwise, in 14 days the issue will be moved to P3.
Please see https://beam.apache.org/contribute/jira-priorities/ for a detailed explanation of what these priorities mean.
> apache beam is not able to find its library when running via DataflowRunner
> ---------------------------------------------------------------------------
>
> Key: BEAM-12779
> URL: https://issues.apache.org/jira/browse/BEAM-12779
> Project: Beam
> Issue Type: Bug
> Components: beam-model
> Affects Versions: 2.31.0
> Environment: (venv) λ python --version
> Python 3.6.1 :: Anaconda 4.4.0 (64-bit)
> (venv) λ pip show apache-beam
> Name: apache-beam
> Version: 2.31.0
> Summary: Apache Beam SDK for Python
> Home-page: https://beam.apache.org
> Author: Apache Software Foundation
> Author-email: dev@beam.apache.org
> License: Apache License, Version 2.0
> Location: venv\lib\site-packages
> Requires: pydot, grpcio, avro-python3, protobuf, dill, python-dateutil, hdfs, httplib2, requests, typing-extensions, future, numpy, pytz, pyarrow, dataclasses, pymongo, oauth2client, crcmod, fastavro
> Required-by: chris-dataflow
> Reporter: Chris C GUO
> Priority: P2
> Labels: stale-P2
>
> INFO:apache_beam.runners.portability.stager:Downloading source distribution of the SDK from PyPi
> INFO:apache_beam.runners.portability.stager:Executing command: ['python.exe', '-m', 'pip', 'download', '--dest', 'AppData\\Local\\Temp\\tmptb7pff18', 'apache-beam==2.31.0', '--no-deps', '--no-binary', ':all:']
>
> ERROR: Could not find a version that satisfies the requirement apache-beam==2.31.0 (from versions: none)
> ERROR: No matching distribution found for apache-beam==2.31.0
> Traceback (most recent call last):
> File "XXXXX\lib\site-packages\apache_beam\utils\processes.py", line 89, in check_output
> out = subprocess.check_output(*args, **kwargs)
> File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 336, in check_output
> **kwargs).stdout
> File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 418, in run
> output=stdout, stderr=stderr)
> subprocess.CalledProcessError: Command '['XXXXX\\venv\\Scripts\\python.exe', '-m', 'pip', 'download', '--dest', 'XXXXX\\AppData\\Local\\Temp\\tmptb7pff18', 'apache-beam==2.31.0', '--no-deps', '--no-binary', ':all:']' returned non-zero exit status 1.
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "rdl_test.py", line 118, in <module>
> pipeline_args,
> File "rdl_test.py", line 82, in run
> | "Write to GCS" >> ParDo(WriteToGCS(output_path))
> File "XXXXX\lib\site-packages\apache_beam\pipeline.py", line 585, in __exit__
> self.result = self.run()
> File "XXXXX\lib\site-packages\apache_beam\pipeline.py", line 540, in run
> self._options).run(False)
> File "XXXXX\lib\site-packages\apache_beam\pipeline.py", line 564, in run
> return self.runner.run_pipeline(self, self._options)
> File "XXXXX\lib\site-packages\apache_beam\runners\dataflow\dataflow_runner.py", line 454, in run_pipeline
> artifacts=environments.python_sdk_dependencies(options),
> File "XXXXX\lib\site-packages\apache_beam\transforms\environments.py", line 806, in python_sdk_dependencies
> skip_prestaged_dependencies=skip_prestaged_dependencies)
> File "XXXXX\lib\site-packages\apache_beam\runners\portability\stager.py", line 285, in create_job_resources
> Stager._create_beam_sdk(sdk_remote_location, temp_dir))
> File "XXXXX\lib\site-packages\apache_beam\runners\portability\stager.py", line 712, in _create_beam_sdk
> sdk_local_file = Stager._download_pypi_sdk_package(temp_dir)
> File "XXXXX\lib\site-packages\apache_beam\runners\portability\stager.py", line 826, in _download_pypi_sdk_package
> processes.check_output(cmd_args)
> File "XXXXX\lib\site-packages\apache_beam\utils\processes.py", line 97, in check_output
> .format(traceback.format_exc(), args[0][6], error.output))
> RuntimeError: Full traceback: Traceback (most recent call last):
> File "XXXXX\lib\site-packages\apache_beam\utils\processes.py", line 89, in check_output
> out = subprocess.check_output(*args, **kwargs)
> File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 336, in check_output
> **kwargs).stdout
> File "C:\ProgramData\Anaconda3\lib\subprocess.py", line 418, in run
> output=stdout, stderr=stderr)
> subprocess.CalledProcessError: Command '['XXXXX\\venv\\Scripts\\python.exe', '-m', 'pip', 'download', '--dest', 'XXXXX\\AppData\\Local\\Temp\\tmptb7pff18', 'apache-beam==2.31.0', '--no-deps', '--no-binary', ':all:']' returned non-zero exit status 1.
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)