You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2019/09/10 00:51:00 UTC
[jira] [Work logged] (BEAM-8119) The python RC validation script
need cleanup the pubsub resources after finish.
[ https://issues.apache.org/jira/browse/BEAM-8119?focusedWorklogId=309420&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-309420 ]
ASF GitHub Bot logged work on BEAM-8119:
----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Sep/19 00:50
Start Date: 10/Sep/19 00:50
Worklog Time Spent: 10m
Work Description: markflyhigh commented on pull request #9534: [BEAM-8119] Properly cleanup Pubsub in release script
URL: https://github.com/apache/beam/pull/9534
Unexpected script failure will leak pubsub topics and fail the next run if no manual cleanup. We want to do cleanup every time before create the topics and also do cleanup when script exists.
Tested manually.
+R: @yifanzou
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 309420)
Remaining Estimate: 0h
Time Spent: 10m
> The python RC validation script need cleanup the pubsub resources after finish.
> -------------------------------------------------------------------------------
>
> Key: BEAM-8119
> URL: https://issues.apache.org/jira/browse/BEAM-8119
> Project: Beam
> Issue Type: Sub-task
> Components: testing
> Reporter: yifan zou
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> * Python staging artifacts were moved to ‘python/’ directory. This needs to be fixed in the script, otherwise the script won’t download the sdks and hash files correctly.
> * If the test fails, it sometimes left some died pubsub topics. This might cause failures in the reruns since the resources already exist:
> * ERROR: Failed to create topic [projects/apache-beam-testing/topics/wordstream-python-topic-1]: Resource already exists in the project (resource=wordstream-python-topic-1).
> * *Workaround*: I manually deleted the topics using cmd: gcloud alpha pubsub topics delete wordstream-python-topic-1
> * TODO:
> ** Change the download directory in the script.
> ** Add cleanup if the job was interrupted.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)