You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Ankur Goenka (JIRA)" <ji...@apache.org> on 2018/10/03 00:44:00 UTC
[jira] [Resolved] (BEAM-5190) Python pipeline options are not
picked correctly by PortableRunner
[ https://issues.apache.org/jira/browse/BEAM-5190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ankur Goenka resolved BEAM-5190.
--------------------------------
Resolution: Fixed
Fix Version/s: 2.8.0
> Python pipeline options are not picked correctly by PortableRunner
> ------------------------------------------------------------------
>
> Key: BEAM-5190
> URL: https://issues.apache.org/jira/browse/BEAM-5190
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-harness
> Reporter: Ankur Goenka
> Assignee: Ankur Goenka
> Priority: Major
> Fix For: 2.8.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Python SDK worker is deserializing the pipeline options to dictionary instead of PipelineOptions
> Sample log
> [grpc-default-executor-2] INFO sdk_worker_main.main - Python sdk harness started with pipeline_options: \{u'beam:option:flink_master:v1': u'[auto]', u'beam:option:streaming:v1': False, u'beam:option:experiments:v1': [u'beam_fn_api', u'worker_threads=50'], u'beam:option:dry_run:v1': False, u'beam:option:runner:v1': None, u'beam:option:profile_memory:v1': False, u'beam:option:runtime_type_check:v1': False, u'beam:option:region:v1': u'us-central1', u'beam:option:options_id:v1': 1, u'beam:option:no_auth:v1': False, u'beam:option:dataflow_endpoint:v1': u'https://dataflow.googleapis.com', u'beam:option:sdk_location:v1': u'/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/dist/apache-beam-2.7.0.dev0.tar.gz', u'beam:option:direct_runner_use_stacked_bundle:v1': True, u'beam:option:save_main_session:v1': True, u'beam:option:type_check_strictness:v1': u'DEFAULT_TO_ANY', u'beam:option:profile_cpu:v1': False, u'beam:option:job_endpoint:v1': u'localhost:8099', u'beam:option:job_name:v1': u'BeamApp-goenka-0822071645-48ae1008', u'beam:option:temp_location:v1': u'gs://clouddfe-goenka/tmp/', u'beam:option:app_name:v1': None, u'beam:option:project:v1': u'google.com:clouddfe', u'beam:option:pipeline_type_check:v1': True, u'beam:option:staging_location:v1': u'gs://clouddfe-goenka/tmp/staging'}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)