You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/05/20 20:14:03 UTC
[GitHub] [beam] benWize opened a new pull request, #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
benWize opened a new pull request, #17727:
URL: https://github.com/apache/beam/pull/17727
Adds a script to check a Kubernetes cluster for available ports, then uses three available ports to instantiate outside services for the Kafka cluster.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1167996754
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
aaltay commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1146294860
@pabloem - could you please run the seed job for this PR? (I tried using your instructions from your email today but failed to have a successful run.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1165759714
https://ci-beam.apache.org/job/beam_SeedJob/9885/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1172933231
Run Java PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1167994878
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155502300
I'll run a seed job against this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155778420
ok feel free to ping me via google chat or on the asf slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1162052586
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155763979
> let me know if you need any help running the Kafka perftest as well
Thanks, Pablo, indeed, I will need help, the trigger phrase has not added the execution to the Jenkins queue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1156650953
seems to have failed for some reason: https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/2778/console
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1146351526
> sorry - I ran SEed Job again - can you try it once more after this job? https://ci-beam.apache.org/job/beam_SeedJob/9774/
Ok, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1146341544
https://ci-beam.apache.org/job/beam_SeedJob/9772/console
I'll record a quick tutorial later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1146141837
Hi @aaltay, could you help me to run a seed job for this PR to validate my changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1164629769
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1169168611
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1146348170
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1176455515
LGTM thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155725873
> oops I'm glad you saw it. Sorry that I forgot to comment (https://ci-beam.apache.org/job/beam_SeedJob/9830/)
I think another [seed job](https://ci-beam.apache.org/job/beam_SeedJob/9831/) overwrote the target job, before the [execution](https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/2777/) that I triggered. We'll need to rerun the seed job.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1133315792
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155761797
let me know if you need any help running the Kafka perftest as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1171603336
Run Java PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1172933278
Run Python PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
aaltay commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1146293085
Run Seed Job
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155499321
@aaltay could you help me to run a seed job for this PR?
cc: @pabloem @kennknowles
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1168008084
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on a diff in pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on code in PR #17727:
URL: https://github.com/apache/beam/pull/17727#discussion_r911294857
##########
.test-infra/jenkins/job_PerformanceTests_KafkaIO_IT.groovy:
##########
@@ -21,23 +21,37 @@ import Kubernetes
import InfluxDBCredentialsHelper
String jobName = "beam_PerformanceTests_Kafka_IO"
+String HIGH_RANGE_PORT = "32767"
job(jobName) {
- common.setTopLevelMainJobProperties(delegate)
- // TODO(https://github.com/apache/beam/issues/20333): Re-enable once fixed.
- // common.setAutoJob(delegate, 'H H/6 * * *')
- // common.enablePhraseTriggeringFromPullRequest(
- // delegate,
- // 'Java KafkaIO Performance Test',
- // 'Run Java KafkaIO Performance Test')
+ common.setTopLevelMainJobProperties(delegate, 'master', 120)
+ common.setAutoJob(delegate, 'H H/6 * * *')
+ common.enablePhraseTriggeringFromPullRequest(
+ delegate,
+ 'Java KafkaIO Performance Test',
+ 'Run Java KafkaIO Performance Test')
InfluxDBCredentialsHelper.useCredentials(delegate)
String namespace = common.getKubernetesNamespace(jobName)
String kubeconfig = common.getKubeconfigLocationForNamespace(namespace)
Kubernetes k8s = Kubernetes.create(delegate, kubeconfig, namespace)
- k8s.apply(common.makePathAbsolute("src/.test-infra/kubernetes/kafka-cluster"))
+ String kafkaDir = common.makePathAbsolute("src/.test-infra/kubernetes/kafka-cluster")
+ String kafkaTopicJob="job.batch/kafka-config-eff079ec"
+
+ // Select available ports for services and avoid collisions
+ steps {
+ String[] configuredPorts = ["32400", "32401", "32402"]
+ (0..2).each { service ->
+ k8s.availablePort(service == 0 ? configuredPorts[service] : "\$KAFKA_SERVICE_PORT_${service-1}",
+ HIGH_RANGE_PORT, "KAFKA_SERVICE_PORT_$service")
+ shell("sed -i -e s/${configuredPorts[service]}/\$KAFKA_SERVICE_PORT_$service/ \
+ ${kafkaDir}/04-outside-services/outside-${service}.yml")
+ }
+ }
Review Comment:
Hi @pablo, I added some comments for describing the requested parts in the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1165947417
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1165980650
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
aaltay commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1165167430
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1133315808
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155717878
oops I'm glad you saw it. Sorry that I forgot to comment (https://ci-beam.apache.org/job/beam_SeedJob/9830/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155764602
We could try after the seed job finishes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155803475
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1159282696
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155730184
Running the new seed jmob: https://ci-beam.apache.org/job/beam_SeedJob/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155752287
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1149242936
@pabloem could you help me to rerun a seed job?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1156684108
> seems to have failed for some reason: https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/2778/console
Yes, for some reason it didn't replace the available ports on the config files, I will take another look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
aaltay commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1165167602
I believe these triggers are disabled and you need to trigger from the jenkins UI. ( @kileys could help)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1176457688
Thank you, @pabloem!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1170396487
Run SQL PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1170314314
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1170464855
R: @pabloem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1159277873
https://ci-beam.apache.org/job/beam_SeedJob/9854/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1161901713
seed job: https://ci-beam.apache.org/job/beam_SeedJob/9870/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1161941735
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1133315785
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1133315806
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1146356790
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1173932983
Run Java PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem merged pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem merged PR #17727:
URL: https://github.com/apache/beam/pull/17727
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1159251762
# [Codecov](https://codecov.io/gh/apache/beam/pull/17727?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#17727](https://codecov.io/gh/apache/beam/pull/17727?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (91b42fa) into [master](https://codecov.io/gh/apache/beam/commit/bcdc5392c2175a48c9c4f75bf5d3b57a4d15ac85?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bcdc539) will **decrease** coverage by `0.15%`.
> The diff coverage is `n/a`.
> :exclamation: Current head 91b42fa differs from pull request most recent head 8716441. Consider uploading reports for the commit 8716441 to get more accurate results
```diff
@@ Coverage Diff @@
## master #17727 +/- ##
==========================================
- Coverage 74.12% 73.97% -0.16%
==========================================
Files 702 702
Lines 93667 92845 -822
==========================================
- Hits 69431 68680 -751
+ Misses 22981 22910 -71
Partials 1255 1255
```
| Flag | Coverage Δ | |
|---|---|---|
| python | `83.57% <ø> (-0.10%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/17727?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/17727/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | `88.09% <0.00%> (-4.77%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/core.py](https://codecov.io/gh/apache/beam/pull/17727/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9jb3JlLnB5) | `92.57% <0.00%> (-1.72%)` | :arrow_down: |
| [...thon/apache\_beam/ml/inference/sklearn\_inference.py](https://codecov.io/gh/apache/beam/pull/17727/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvaW5mZXJlbmNlL3NrbGVhcm5faW5mZXJlbmNlLnB5) | `95.23% <0.00%> (-1.58%)` | :arrow_down: |
| [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/17727/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | `90.06% <0.00%> (-1.33%)` | :arrow_down: |
| [...cp/internal/clients/bigquery/bigquery\_v2\_client.py](https://codecov.io/gh/apache/beam/pull/17727/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2ludGVybmFsL2NsaWVudHMvYmlncXVlcnkvYmlncXVlcnlfdjJfY2xpZW50LnB5) | `66.82% <0.00%> (-0.97%)` | :arrow_down: |
| [sdks/python/apache\_beam/ml/inference/base.py](https://codecov.io/gh/apache/beam/pull/17727/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvaW5mZXJlbmNlL2Jhc2UucHk=) | `95.29% <0.00%> (-0.46%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/combiners.py](https://codecov.io/gh/apache/beam/pull/17727/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9jb21iaW5lcnMucHk=) | `93.05% <0.00%> (-0.39%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/17727/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.94% <0.00%> (-0.16%)` | :arrow_down: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/17727/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.54% <0.00%> (-0.13%)` | :arrow_down: |
| [sdks/python/apache\_beam/ml/inference/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/17727/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvaW5mZXJlbmNlL19faW5pdF9fLnB5) | `100.00% <0.00%> (ø)` | |
| ... and [5 more](https://codecov.io/gh/apache/beam/pull/17727/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/17727?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/17727?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bcdc539...8716441](https://codecov.io/gh/apache/beam/pull/17727?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1167994385
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1170396132
Successful execution https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/2789/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1165813964
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1172933261
Run Java_PVR_Flink_Batch PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on a diff in pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on code in PR #17727:
URL: https://github.com/apache/beam/pull/17727#discussion_r911294857
##########
.test-infra/jenkins/job_PerformanceTests_KafkaIO_IT.groovy:
##########
@@ -21,23 +21,37 @@ import Kubernetes
import InfluxDBCredentialsHelper
String jobName = "beam_PerformanceTests_Kafka_IO"
+String HIGH_RANGE_PORT = "32767"
job(jobName) {
- common.setTopLevelMainJobProperties(delegate)
- // TODO(https://github.com/apache/beam/issues/20333): Re-enable once fixed.
- // common.setAutoJob(delegate, 'H H/6 * * *')
- // common.enablePhraseTriggeringFromPullRequest(
- // delegate,
- // 'Java KafkaIO Performance Test',
- // 'Run Java KafkaIO Performance Test')
+ common.setTopLevelMainJobProperties(delegate, 'master', 120)
+ common.setAutoJob(delegate, 'H H/6 * * *')
+ common.enablePhraseTriggeringFromPullRequest(
+ delegate,
+ 'Java KafkaIO Performance Test',
+ 'Run Java KafkaIO Performance Test')
InfluxDBCredentialsHelper.useCredentials(delegate)
String namespace = common.getKubernetesNamespace(jobName)
String kubeconfig = common.getKubeconfigLocationForNamespace(namespace)
Kubernetes k8s = Kubernetes.create(delegate, kubeconfig, namespace)
- k8s.apply(common.makePathAbsolute("src/.test-infra/kubernetes/kafka-cluster"))
+ String kafkaDir = common.makePathAbsolute("src/.test-infra/kubernetes/kafka-cluster")
+ String kafkaTopicJob="job.batch/kafka-config-eff079ec"
+
+ // Select available ports for services and avoid collisions
+ steps {
+ String[] configuredPorts = ["32400", "32401", "32402"]
+ (0..2).each { service ->
+ k8s.availablePort(service == 0 ? configuredPorts[service] : "\$KAFKA_SERVICE_PORT_${service-1}",
+ HIGH_RANGE_PORT, "KAFKA_SERVICE_PORT_$service")
+ shell("sed -i -e s/${configuredPorts[service]}/\$KAFKA_SERVICE_PORT_$service/ \
+ ${kafkaDir}/04-outside-services/outside-${service}.yml")
+ }
+ }
Review Comment:
Hi @pabloem, I added some comments for describing the requested parts in the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on a diff in pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on code in PR #17727:
URL: https://github.com/apache/beam/pull/17727#discussion_r911301717
##########
.test-infra/jenkins/job_PerformanceTests_KafkaIO_IT.groovy:
##########
@@ -21,23 +21,37 @@ import Kubernetes
import InfluxDBCredentialsHelper
String jobName = "beam_PerformanceTests_Kafka_IO"
+String HIGH_RANGE_PORT = "32767"
job(jobName) {
- common.setTopLevelMainJobProperties(delegate)
- // TODO(https://github.com/apache/beam/issues/20333): Re-enable once fixed.
- // common.setAutoJob(delegate, 'H H/6 * * *')
- // common.enablePhraseTriggeringFromPullRequest(
- // delegate,
- // 'Java KafkaIO Performance Test',
- // 'Run Java KafkaIO Performance Test')
+ common.setTopLevelMainJobProperties(delegate, 'master', 120)
+ common.setAutoJob(delegate, 'H H/6 * * *')
+ common.enablePhraseTriggeringFromPullRequest(
+ delegate,
+ 'Java KafkaIO Performance Test',
+ 'Run Java KafkaIO Performance Test')
InfluxDBCredentialsHelper.useCredentials(delegate)
String namespace = common.getKubernetesNamespace(jobName)
String kubeconfig = common.getKubeconfigLocationForNamespace(namespace)
Kubernetes k8s = Kubernetes.create(delegate, kubeconfig, namespace)
- k8s.apply(common.makePathAbsolute("src/.test-infra/kubernetes/kafka-cluster"))
+ String kafkaDir = common.makePathAbsolute("src/.test-infra/kubernetes/kafka-cluster")
+ String kafkaTopicJob="job.batch/kafka-config-eff079ec"
Review Comment:
Fixed it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1170192546
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on a diff in pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on code in PR #17727:
URL: https://github.com/apache/beam/pull/17727#discussion_r910413494
##########
.test-infra/jenkins/Kubernetes.groovy:
##########
@@ -132,4 +132,28 @@ class Kubernetes {
}
}
}
+
+ /**
+ * Specifies steps that will return an available port on Kubernetes cluster
+ *
+ * @param lowRangePort - low range port to be used
+ * @param highRangePort - high range port to be used
+ * @param referenceName - name of the environment variable
+ */
+ void availablePort(String lowRangePort, String highRangePort, String referenceName) {
Review Comment:
is there a way to log what's happening so we can debug in case of failures?
##########
.test-infra/jenkins/job_PerformanceTests_KafkaIO_IT.groovy:
##########
@@ -21,23 +21,37 @@ import Kubernetes
import InfluxDBCredentialsHelper
String jobName = "beam_PerformanceTests_Kafka_IO"
+String HIGH_RANGE_PORT = "32767"
job(jobName) {
- common.setTopLevelMainJobProperties(delegate)
- // TODO(https://github.com/apache/beam/issues/20333): Re-enable once fixed.
- // common.setAutoJob(delegate, 'H H/6 * * *')
- // common.enablePhraseTriggeringFromPullRequest(
- // delegate,
- // 'Java KafkaIO Performance Test',
- // 'Run Java KafkaIO Performance Test')
+ common.setTopLevelMainJobProperties(delegate, 'master', 120)
+ common.setAutoJob(delegate, 'H H/6 * * *')
+ common.enablePhraseTriggeringFromPullRequest(
+ delegate,
+ 'Java KafkaIO Performance Test',
+ 'Run Java KafkaIO Performance Test')
InfluxDBCredentialsHelper.useCredentials(delegate)
String namespace = common.getKubernetesNamespace(jobName)
String kubeconfig = common.getKubeconfigLocationForNamespace(namespace)
Kubernetes k8s = Kubernetes.create(delegate, kubeconfig, namespace)
- k8s.apply(common.makePathAbsolute("src/.test-infra/kubernetes/kafka-cluster"))
+ String kafkaDir = common.makePathAbsolute("src/.test-infra/kubernetes/kafka-cluster")
+ String kafkaTopicJob="job.batch/kafka-config-eff079ec"
+
+ // Select available ports for services and avoid collisions
+ steps {
+ String[] configuredPorts = ["32400", "32401", "32402"]
+ (0..2).each { service ->
+ k8s.availablePort(service == 0 ? configuredPorts[service] : "\$KAFKA_SERVICE_PORT_${service-1}",
+ HIGH_RANGE_PORT, "KAFKA_SERVICE_PORT_$service")
+ shell("sed -i -e s/${configuredPorts[service]}/\$KAFKA_SERVICE_PORT_$service/ \
+ ${kafkaDir}/04-outside-services/outside-${service}.yml")
+ }
+ }
Review Comment:
Can you add a comment so I can understand what this does? To be honest, I don't know where `KAFKA_SERVICE_PORT_` and other special variables are coming from.
Please also document this in the confluence wiki.
##########
.test-infra/jenkins/job_PerformanceTests_KafkaIO_IT.groovy:
##########
@@ -21,23 +21,37 @@ import Kubernetes
import InfluxDBCredentialsHelper
String jobName = "beam_PerformanceTests_Kafka_IO"
+String HIGH_RANGE_PORT = "32767"
job(jobName) {
- common.setTopLevelMainJobProperties(delegate)
- // TODO(https://github.com/apache/beam/issues/20333): Re-enable once fixed.
- // common.setAutoJob(delegate, 'H H/6 * * *')
- // common.enablePhraseTriggeringFromPullRequest(
- // delegate,
- // 'Java KafkaIO Performance Test',
- // 'Run Java KafkaIO Performance Test')
+ common.setTopLevelMainJobProperties(delegate, 'master', 120)
+ common.setAutoJob(delegate, 'H H/6 * * *')
+ common.enablePhraseTriggeringFromPullRequest(
+ delegate,
+ 'Java KafkaIO Performance Test',
+ 'Run Java KafkaIO Performance Test')
InfluxDBCredentialsHelper.useCredentials(delegate)
String namespace = common.getKubernetesNamespace(jobName)
String kubeconfig = common.getKubeconfigLocationForNamespace(namespace)
Kubernetes k8s = Kubernetes.create(delegate, kubeconfig, namespace)
- k8s.apply(common.makePathAbsolute("src/.test-infra/kubernetes/kafka-cluster"))
+ String kafkaDir = common.makePathAbsolute("src/.test-infra/kubernetes/kafka-cluster")
+ String kafkaTopicJob="job.batch/kafka-config-eff079ec"
Review Comment:
```suggestion
String kafkaTopicJob = "job.batch/kafka-config-eff079ec"
```
nitpicking : )
##########
.test-infra/jenkins/job_PerformanceTests_KafkaIO_IT.groovy:
##########
@@ -21,23 +21,37 @@ import Kubernetes
import InfluxDBCredentialsHelper
String jobName = "beam_PerformanceTests_Kafka_IO"
+String HIGH_RANGE_PORT = "32767"
job(jobName) {
- common.setTopLevelMainJobProperties(delegate)
- // TODO(https://github.com/apache/beam/issues/20333): Re-enable once fixed.
- // common.setAutoJob(delegate, 'H H/6 * * *')
- // common.enablePhraseTriggeringFromPullRequest(
- // delegate,
- // 'Java KafkaIO Performance Test',
- // 'Run Java KafkaIO Performance Test')
+ common.setTopLevelMainJobProperties(delegate, 'master', 120)
+ common.setAutoJob(delegate, 'H H/6 * * *')
+ common.enablePhraseTriggeringFromPullRequest(
+ delegate,
+ 'Java KafkaIO Performance Test',
+ 'Run Java KafkaIO Performance Test')
InfluxDBCredentialsHelper.useCredentials(delegate)
String namespace = common.getKubernetesNamespace(jobName)
String kubeconfig = common.getKubeconfigLocationForNamespace(namespace)
Kubernetes k8s = Kubernetes.create(delegate, kubeconfig, namespace)
- k8s.apply(common.makePathAbsolute("src/.test-infra/kubernetes/kafka-cluster"))
+ String kafkaDir = common.makePathAbsolute("src/.test-infra/kubernetes/kafka-cluster")
+ String kafkaTopicJob="job.batch/kafka-config-eff079ec"
+
+ // Select available ports for services and avoid collisions
+ steps {
+ String[] configuredPorts = ["32400", "32401", "32402"]
+ (0..2).each { service ->
+ k8s.availablePort(service == 0 ? configuredPorts[service] : "\$KAFKA_SERVICE_PORT_${service-1}",
+ HIGH_RANGE_PORT, "KAFKA_SERVICE_PORT_$service")
+ shell("sed -i -e s/${configuredPorts[service]}/\$KAFKA_SERVICE_PORT_$service/ \
+ ${kafkaDir}/04-outside-services/outside-${service}.yml")
+ }
+ }
Review Comment:
Please add a comment at the top describing what resources are used by this test and what the test does.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on a diff in pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on code in PR #17727:
URL: https://github.com/apache/beam/pull/17727#discussion_r911298468
##########
.test-infra/jenkins/Kubernetes.groovy:
##########
@@ -132,4 +132,28 @@ class Kubernetes {
}
}
}
+
+ /**
+ * Specifies steps that will return an available port on Kubernetes cluster
+ *
+ * @param lowRangePort - low range port to be used
+ * @param highRangePort - high range port to be used
+ * @param referenceName - name of the environment variable
+ */
+ void availablePort(String lowRangePort, String highRangePort, String referenceName) {
Review Comment:
This function already outputs to the log console https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/2789/consoleFull
`13:01:36 + getAvailablePort 32400 32767
13:01:36 + local lowRangePort=32401
13:01:36 + local highRangePort=32767
13:01:36 + local used=false
13:01:36 + local 'command=kubectl --kubeconfig=/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Kafka_IO/config-beam-performancetests-kafka-io-2789 --namespace=beam-performancetests-kafka-io-2789 get svc --all-namespaces -o go-template='\''{{range .items}}{{range.spec.ports}}{{if .nodePort}}{{.nodePort}}{{"\n"}}{{end}}{{end}}{{end}}'\'''`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155611699
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1155740989
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1133315788
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1162056919
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1165788975
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] benWize commented on pull request #17727: [BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
benWize commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1165814440
Run Java KafkaIO Performance Test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #17727: [WIP][BEAM-9482] Fix "provided port is already allocated" for PerformanceTestsKafka
Posted by GitBox <gi...@apache.org>.
pabloem commented on PR #17727:
URL: https://github.com/apache/beam/pull/17727#issuecomment-1146350371
sorry - I ran SEed Job again - can you try it once more after this job? https://ci-beam.apache.org/job/beam_SeedJob/9774/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org