You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "pgaref (via GitHub)" <gi...@apache.org> on 2023/05/17 19:28:54 UTC

[GitHub] [flink] pgaref opened a new pull request, #22603: [FLINK-32123][e2e] Fix E2E kafka download timeout

pgaref opened a new pull request, #22603:
URL: https://github.com/apache/flink/pull/22603

   https://issues.apache.org/jira/browse/FLINK-32123
   
   For the past few hours, E2E tests fail with: `Avro Confluent Schema Registry nightly end-to-end test failed after 9 minutes and 53 seconds! Test exited with exit code 1`
   
   Looks like https://archive.apache.org/dist/kafka/  mirror is overloaded – download locally took more than 30min
   
   This PR switches to  https://downloads.apache.org/ mirror (locally take a few minutes)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] pgaref commented on pull request #22603: [FLINK-32123][e2e] Fix E2E kafka download timeout

Posted by "pgaref (via GitHub)" <gi...@apache.org>.
pgaref commented on PR #22603:
URL: https://github.com/apache/flink/pull/22603#issuecomment-1552062205

   @flinkbot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] pgaref commented on a diff in pull request #22603: [FLINK-32123][e2e] Fix E2E kafka download timeout

Posted by "pgaref (via GitHub)" <gi...@apache.org>.
pgaref commented on code in PR #22603:
URL: https://github.com/apache/flink/pull/22603#discussion_r1197420627


##########
flink-end-to-end-tests/test-scripts/kafka-common.sh:
##########
@@ -37,7 +37,7 @@ MAX_RETRY_SECONDS=120
 function setup_kafka_dist {
   # download Kafka
   mkdir -p $TEST_DATA_DIR
-  KAFKA_URL="https://archive.apache.org/dist/kafka/$KAFKA_VERSION/kafka_2.12-$KAFKA_VERSION.tgz"
+  KAFKA_URL="https://downloads.apache.org/kafka/$KAFKA_VERSION/kafka_2.12-$KAFKA_VERSION.tgz"

Review Comment:
   I believe we are keeping up with Kafka upgrades anyway, as its a fundamental component. If we want to stick to older version in the future we can always switch back to the archive (hoping the slow-down is resolved, or bumping the timeout)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] dmvk commented on pull request #22603: [FLINK-32123][e2e] Fix E2E kafka download timeout

Posted by "dmvk (via GitHub)" <gi...@apache.org>.
dmvk commented on PR #22603:
URL: https://github.com/apache/flink/pull/22603#issuecomment-1552504902

   I've opened an issue with INFRA [1]. Let's give them some time to respond. If this is not fixed by noon CET, I will merge this as a temporary hotfix and we'll revert it later.
   
   [1]


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] reswqa commented on a diff in pull request #22603: [FLINK-32123][e2e] Fix E2E kafka download timeout

Posted by "reswqa (via GitHub)" <gi...@apache.org>.
reswqa commented on code in PR #22603:
URL: https://github.com/apache/flink/pull/22603#discussion_r1197414843


##########
flink-end-to-end-tests/test-scripts/kafka-common.sh:
##########
@@ -37,7 +37,7 @@ MAX_RETRY_SECONDS=120
 function setup_kafka_dist {
   # download Kafka
   mkdir -p $TEST_DATA_DIR
-  KAFKA_URL="https://archive.apache.org/dist/kafka/$KAFKA_VERSION/kafka_2.12-$KAFKA_VERSION.tgz"
+  KAFKA_URL="https://downloads.apache.org/kafka/$KAFKA_VERSION/kafka_2.12-$KAFKA_VERSION.tgz"

Review Comment:
   Does this mean that when the Kafka version we are using is no longer recommended, we must forcibly upgrade it. I'm not sure if we're currently syncing updates.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] dmvk merged pull request #22603: [FLINK-32123][e2e] Fix E2E kafka download timeout

Posted by "dmvk (via GitHub)" <gi...@apache.org>.
dmvk merged PR #22603:
URL: https://github.com/apache/flink/pull/22603


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] dmvk commented on a diff in pull request #22603: [FLINK-32123][e2e] Fix E2E kafka download timeout

Posted by "dmvk (via GitHub)" <gi...@apache.org>.
dmvk commented on code in PR #22603:
URL: https://github.com/apache/flink/pull/22603#discussion_r1197426232


##########
flink-end-to-end-tests/test-scripts/kafka-common.sh:
##########
@@ -37,7 +37,7 @@ MAX_RETRY_SECONDS=120
 function setup_kafka_dist {
   # download Kafka
   mkdir -p $TEST_DATA_DIR
-  KAFKA_URL="https://archive.apache.org/dist/kafka/$KAFKA_VERSION/kafka_2.12-$KAFKA_VERSION.tgz"
+  KAFKA_URL="https://downloads.apache.org/kafka/$KAFKA_VERSION/kafka_2.12-$KAFKA_VERSION.tgz"

Review Comment:
   > I believe we are keeping up with Kafka upgrades anyway, as its a fundamental component.
   
   An upgrade must be deliberate on the Flink side; for example, if Kafka pushes a faulty release, we might want to skip it, but the older one would no longer be available.
   
   Also, we need to have some headroom for the upgrade without being stressed about all builds suddenly failing.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #22603: [FLINK-32123][e2e] Fix E2E kafka download timeout

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #22603:
URL: https://github.com/apache/flink/pull/22603#issuecomment-1551947956

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "66a1e4729adf5ba5edd75e7a9bc9e7b90c430ea3",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "66a1e4729adf5ba5edd75e7a9bc9e7b90c430ea3",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 66a1e4729adf5ba5edd75e7a9bc9e7b90c430ea3 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org