You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Juta Staes (JIRA)" <ji...@apache.org> on 2019/04/09 16:10:00 UTC
[jira] [Closed] (BEAM-6870) python 3 test_hourly_team_score_it
fails with bigquery job id already exists
[ https://issues.apache.org/jira/browse/BEAM-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Juta Staes closed BEAM-6870.
----------------------------
Resolution: Cannot Reproduce
Fix Version/s: Not applicable
> python 3 test_hourly_team_score_it fails with bigquery job id already exists
> ----------------------------------------------------------------------------
>
> Key: BEAM-6870
> URL: https://issues.apache.org/jira/browse/BEAM-6870
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
> Reporter: Juta Staes
> Priority: Major
> Fix For: Not applicable
>
>
> This IT test on python 3 fails with "Already exists job ..."
> {code:java}
> 14:42:03 ======================================================================
> 14:42:03 ERROR: test_hourly_team_score_it (apache_beam.examples.complete.game.hourly_team_score_it_test.HourlyTeamScoreIT)
> 14:42:03 ----------------------------------------------------------------------
> 14:42:03 Traceback (most recent call last):
> 14:42:03 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify_PR/src/sdks/python/apache_beam/examples/complete/game/hourly_team_score_it_test.py", line 89, in test_hourly_team_score_it
> 14:42:03 self.test_pipeline.get_full_options_as_args(**extra_opts))
> 14:42:03 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify_PR/src/sdks/python/apache_beam/examples/complete/game/hourly_team_score.py", line 295, in run
> 14:42:03 }, options.view_as(GoogleCloudOptions).project))
> 14:42:03 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify_PR/src/sdks/python/apache_beam/pipeline.py", line 426, in __exit__
> 14:42:03 self.run().wait_until_finish()
> 14:42:03 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify_PR/src/sdks/python/apache_beam/pipeline.py", line 406, in run
> 14:42:03 self._options).run(False)
> 14:42:03 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify_PR/src/sdks/python/apache_beam/pipeline.py", line 419, in run
> 14:42:03 return self.runner.run_pipeline(self, self._options)
> 14:42:03 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify_PR/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py", line 64, in run_pipeline
> 14:42:03 self.result.wait_until_finish(duration=wait_duration)
> 14:42:03 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python3_Verify_PR/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py", line 1217, in wait_until_finish
> 14:42:03 (self.state, getattr(self._runner, 'last_error_msg', None)), self)
> 14:42:03 apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
> 14:42:03 Traceback (most recent call last):
> 14:42:03 File "apache_beam/runners/common.py", line 732, in apache_beam.runners.common.DoFnRunner.process
> 14:42:03 File "apache_beam/runners/common.py", line 555, in apache_beam.runners.common.PerWindowInvoker.invoke_process
> 14:42:03 File "apache_beam/runners/common.py", line 624, in apache_beam.runners.common.PerWindowInvoker._invoke_per_window
> 14:42:03 File "apache_beam/runners/common.py", line 816, in apache_beam.runners.common._OutputProcessor.process_outputs
> 14:42:03 File "apache_beam/runners/common.py", line 831, in apache_beam.runners.common._OutputProcessor.process_outputs
> 14:42:03 File "/usr/local/lib/python3.5/site-packages/apache_beam/io/gcp/bigquery_file_loads.py", line 385, in process
> 14:42:03 create_disposition=self.create_disposition)
> 14:42:03 File "/usr/local/lib/python3.5/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 560, in perform_load_job
> 14:42:03 write_disposition=write_disposition)
> 14:42:03 File "/usr/local/lib/python3.5/site-packages/apache_beam/utils/retry.py", line 195, in wrapper
> 14:42:03 return fun(*args, **kwargs)
> 14:42:03 File "/usr/local/lib/python3.5/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 327, in _insert_load_job
> 14:42:03 response = self.client.jobs.Insert(request)
> 14:42:03 File "/usr/local/lib/python3.5/site-packages/apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 342, in Insert
> 14:42:03 upload=upload, upload_config=upload_config)
> 14:42:03 File "/usr/local/lib/python3.5/site-packages/apitools/base/py/base_api.py", line 731, in _RunMethod
> 14:42:03 return self.ProcessHttpResponse(method_config, http_response, request)
> 14:42:03 File "/usr/local/lib/python3.5/site-packages/apitools/base/py/base_api.py", line 737, in ProcessHttpResponse
> 14:42:03 self.__ProcessHttpResponse(method_config, http_response, request))
> 14:42:03 File "/usr/local/lib/python3.5/site-packages/apitools/base/py/base_api.py", line 604, in __ProcessHttpResponse
> 14:42:03 http_response, method_config=method_config, request=request)
> 14:42:03 apitools.base.py.exceptions.HttpConflictError: HttpError accessing <https://www.googleapis.com/bigquery/v2/projects/apache-beam-testing/jobs?alt=json>: response: <{'status': '409', 'content-type': 'application/json; charset=UTF-8', 'x-content-type-options': 'nosniff', 'transfer-encoding': 'chunked', 'cache-control': 'private, max-age=0', '-content-encoding': 'gzip', 'date': 'Wed, 20 Mar 2019 13:29:44 GMT', 'server': 'GSE', 'content-length': '364', 'x-xss-protection': '1; mode=block', 'vary': 'Origin, X-Origin', 'x-frame-options': 'SAMEORIGIN', 'expires': 'Wed, 20 Mar 2019 13:29:44 GMT'}>, content <{
> 14:42:03 "error": {
> 14:42:03 "errors": [
> 14:42:03 {
> 14:42:03 "domain": "global",
> 14:42:03 "reason": "duplicate",
> 14:42:03 "message": "Already Exists: Job apache-beam-testing:US.beam_load_2019_03_20_132147_71_1fea0de5d8e357a03e39484b3ebe0123_0"
> 14:42:03 }
> 14:42:03 ],
> 14:42:03 "code": 409,
> 14:42:03 "message": "Already Exists: Job apache-beam-testing:US.beam_load_2019_03_20_132147_71_1fea0de5d8e357a03e39484b3ebe0123_0"
> 14:42:03 }
> 14:42:03 }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)