You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Udi Meiri (JIRA)" <ji...@apache.org> on 2019/07/12 01:34:00 UTC
[jira] [Closed] (BEAM-3990) Dataflow jobs fail with "KeyError:
'location'" when uploading to GCS
[ https://issues.apache.org/jira/browse/BEAM-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Udi Meiri closed BEAM-3990.
---------------------------
Resolution: Fixed
Fix Version/s: Not applicable
> Dataflow jobs fail with "KeyError: 'location'" when uploading to GCS
> --------------------------------------------------------------------
>
> Key: BEAM-3990
> URL: https://issues.apache.org/jira/browse/BEAM-3990
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Chamikara Jayalath
> Priority: Major
> Fix For: Not applicable
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Some Dataflow jobs are failing due to following error (in worker logs).
>
> Error in _start_upload while inserting file gs://cloud-ml-benchmark-output-us-central/df1-cloudml-benchmark-criteo-small-python-033010274088282-presubmit3/033010274088282/temp/df1-cloudml-benchmark-criteo-small-python-033010274088282-presubmit3.1522430898.446147/dax-tmp-2018-03-30_10_28_40-14595186994726940229-S241-1-dc87ef69274882bf/tmp-dc87ef6927488c5a-shard--try-308ae8b3268d12b2-endshard.avro: Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/gcp/gcsio.py", line 559, in _start_upload self._client.objects.Insert(self._insert_request, upload=self._upload) File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/gcp/internal/clients/storage/storage_v1_client.py", line 971, in Insert download=download) File "/usr/local/lib/python2.7/dist-packages/apitools/base/py/base_api.py", line 706, in _RunMethod http_request, client=self.client) File "/usr/local/lib/python2.7/dist-packages/apitools/base/py/transfer.py", line 860, in InitializeUpload url = [http_response.info|https://www.google.com/url?q=http://http_response.info&sa=D&usg=AFQjCNGvYHYJBb_G4YNo3VvGoqX2Gq-6Yw]['location'] KeyError: 'location'
>
> This seems to be due to [https://github.com/apache/beam/pull/4891.] Possibly storage.StorageV1() cannot be shared across multiple requests without additional fixes.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)