You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Thomas Weise (JIRA)" <ji...@apache.org> on 2018/09/26 01:46:00 UTC
[jira] [Commented] (BEAM-5509) Python pipeline_options doesn't
handle int type
[ https://issues.apache.org/jira/browse/BEAM-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628141#comment-16628141 ]
Thomas Weise commented on BEAM-5509:
------------------------------------
Pass --parallelism=1 and then job_utils.dict_to_struct(options) yields
{code:java}
fields {
key: "beam:option:parallelism:v1"
value {
number_value: 1.0
}
}{code}
Parsing in SDK harness will bark at it:
{code:java}
sdk_worker_main.py: error: argument --parallelism: invalid int value: u'1.0'
[grpc-default-worker-ELG-3-3] DEBUG org.apache.beam.vendor.grpc.v1.io.grpc.netty.NettyServerHandler - [id: 0x284d90f2, L:/127.0.0.1:57436 - R:/127.0.0.1:57442] INBOUND DATA: streamId=1 padding=0 endStream=false length=980 bytes=00000003cf0acc070806120c08ccb8abdd0510e8b5f5e1011a9707507974686f6e2073646b206861726e657373206661696c65643a200a54726163656261636b...
[grpc-default-worker-ELG-3-3] DEBUG org.apache.beam.vendor.grpc.v1.io.grpc.netty.NettyServerHandler - [id: 0x284d90f2, L:/127.0.0.1:57436 - R:/127.0.0.1:57442] INBOUND DATA: streamId=1 padding=0 endStream=true length=0 bytes=
[grpc-default-executor-0] ERROR sdk_worker_main.main - Python sdk harness failed:
Traceback (most recent call last):
File "/Users/tweise/python-ve/beam/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py", line 136, in main
sdk_pipeline_options.get_all_options(drop_default=True))
File "/Users/tweise/python-ve/beam/lib/python2.7/site-packages/apache_beam/options/pipeline_options.py", line 216, in get_all_options
known_args, _ = parser.parse_known_args(self._flags)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1740, in parse_known_args
self.error(str(err))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 2374, in error
self.exit(2, _('%s: error: %s\n') % (self.prog, message))
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 2362, in exit
_sys.exit(status)
SystemExit: 2{code}
> Python pipeline_options doesn't handle int type
> -----------------------------------------------
>
> Key: BEAM-5509
> URL: https://issues.apache.org/jira/browse/BEAM-5509
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Reporter: Thomas Weise
> Assignee: Robert Bradshaw
> Priority: Major
>
> The int option supplied at the command line is turned into a decimal during serialization and then the parser in SDK harness fails to restore it as int.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)