You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by GitBox <gi...@apache.org> on 2023/01/18 16:10:53 UTC
[GitHub] [ignite] shishkovilja opened a new pull request, #10488: IGNITE-18574 CDC: add documentation about Kafka request timeouts
shishkovilja opened a new pull request, #10488:
URL: https://github.com/apache/ignite/pull/10488
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:
### The Contribution Checklist
- [ ] There is a single JIRA ticket related to the pull request.
- [ ] The web-link to the pull request is attached to the JIRA ticket.
- [ ] The JIRA ticket has the _Patch Available_ state.
- [ ] The pull request body describes changes that have been made.
The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
- [ ] The pull request title is treated as the final commit message.
The following pattern must be used: `IGNITE-XXXX Change summary` where `XXXX` - number of JIRA issue.
- [ ] A reviewer has been mentioned through the JIRA comments
(see [the Maintainers list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers))
- [ ] The pull request has been checked by the Teamcity Bot and
the `green visa` attached to the JIRA ticket (see [TC.Bot: Check PR](https://mtcga.gridgain.com/prs.html))
### Notes
- [How to Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
- [Coding abbreviation rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
- [Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
- [Apache Ignite Teamcity Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com _#ignite_ channel.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10488: IGNITE-18574 CDC: add documentation about Kafka request timeouts
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10488:
URL: https://github.com/apache/ignite/pull/10488#issuecomment-1501833684
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10488)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10488&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10488&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10488&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10488&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10488&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] shishkovilja commented on a diff in pull request #10488: IGNITE-18574 CDC: add documentation about Kafka request timeouts
Posted by "shishkovilja (via GitHub)" <gi...@apache.org>.
shishkovilja commented on code in PR #10488:
URL: https://github.com/apache/ignite/pull/10488#discussion_r1161620780
##########
docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc:
##########
@@ -113,6 +113,14 @@ image:../../assets/images/integrations/CDC-ignite2kafka.svg[]
| `kafkaRequestTimeout` | Kafka request timeout in milliseconds. | `3000`
|===
+`kafkaRequestTimeout` property sets how much `IgniteToKafkaCdcStreamer` will wait for `KafkaProducer` to finish request.
+
+NOTE: _Too low value_ of this timeout can lead to frequent failures of `IgniteToKafkaCdcStreamer`
+because of possible delays in network or on Kafka broker.
+Also, proper `KafkaProducer` configuration should be provided, including link:https://kafka.apache.org/documentation/#producerconfigs_request.timeout.ms[request.timeout.ms].
Review Comment:
Good point!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] timoninmaxim commented on a diff in pull request #10488: IGNITE-18574 CDC: add documentation about Kafka request timeouts
Posted by "timoninmaxim (via GitHub)" <gi...@apache.org>.
timoninmaxim commented on code in PR #10488:
URL: https://github.com/apache/ignite/pull/10488#discussion_r1161593746
##########
docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc:
##########
@@ -175,6 +183,16 @@ Kafka to Ignite configuration file should contain the following beans that will
| `threadCount` | Count of threads to proceed consumers. Each thread poll records from dedicated partitions in round-robin manner. | 16
|===
+`kafkaRequestTimeout` property sets timeouts for a following `KafkaConsumer` operations:
+
+- Poll from event and metadata topics.
Review Comment:
Do we actually need list of these operations? Is there any kafka request that doesn't use this timeout?
##########
docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc:
##########
@@ -175,6 +183,16 @@ Kafka to Ignite configuration file should contain the following beans that will
| `threadCount` | Count of threads to proceed consumers. Each thread poll records from dedicated partitions in round-robin manner. | 16
|===
+`kafkaRequestTimeout` property sets timeouts for a following `KafkaConsumer` operations:
+
+- Poll from event and metadata topics.
+- Offsets commit.
+- Closing of a consumer.
+- Offsets request (during metadata updates).
+- Metadata topic partitions request (during metadata updater start).
+
+NOTE: _Too low value_ of `kafkaRequestTimeout` can lead to frequent failures of `kafka-to-ignite.sh` because of possible delays in network or on Kafka broker. Also, proper `KafkaConsumer` configuration should be provided, including link:https://kafka.apache.org/documentation/#consumerconfigs_request.timeout.ms[request.timeout.ms]. For more details you should refer to a link:https://kafka.apache.org/documentation/#configuration[configuration] section of the official Kafka documentation.
Review Comment:
Same comment for font.
##########
docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc:
##########
@@ -113,6 +113,14 @@ image:../../assets/images/integrations/CDC-ignite2kafka.svg[]
| `kafkaRequestTimeout` | Kafka request timeout in milliseconds. | `3000`
|===
+`kafkaRequestTimeout` property sets how much `IgniteToKafkaCdcStreamer` will wait for `KafkaProducer` to finish request.
+
+NOTE: _Too low value_ of this timeout can lead to frequent failures of `IgniteToKafkaCdcStreamer`
Review Comment:
For me, using NOTE is enough for the emphasis. No need in using italic font here.
##########
docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc:
##########
@@ -113,6 +113,14 @@ image:../../assets/images/integrations/CDC-ignite2kafka.svg[]
| `kafkaRequestTimeout` | Kafka request timeout in milliseconds. | `3000`
|===
+`kafkaRequestTimeout` property sets how much `IgniteToKafkaCdcStreamer` will wait for `KafkaProducer` to finish request.
+
+NOTE: _Too low value_ of this timeout can lead to frequent failures of `IgniteToKafkaCdcStreamer`
+because of possible delays in network or on Kafka broker.
+Also, proper `KafkaProducer` configuration should be provided, including link:https://kafka.apache.org/documentation/#producerconfigs_request.timeout.ms[request.timeout.ms].
Review Comment:
What is a purpose of this new line? Should this sentence be part of the NOTE or not? If not, please add additional empty line between the NOTE paragraph and the sentence.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10488: IGNITE-18574 CDC: add documentation about Kafka request timeouts
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10488:
URL: https://github.com/apache/ignite/pull/10488#issuecomment-1501578777
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10488)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10488&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10488&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10488&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10488&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10488&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] shishkovilja commented on a diff in pull request #10488: IGNITE-18574 CDC: add documentation about Kafka request timeouts
Posted by "shishkovilja (via GitHub)" <gi...@apache.org>.
shishkovilja commented on code in PR #10488:
URL: https://github.com/apache/ignite/pull/10488#discussion_r1161620359
##########
docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc:
##########
@@ -113,6 +113,14 @@ image:../../assets/images/integrations/CDC-ignite2kafka.svg[]
| `kafkaRequestTimeout` | Kafka request timeout in milliseconds. | `3000`
|===
+`kafkaRequestTimeout` property sets how much `IgniteToKafkaCdcStreamer` will wait for `KafkaProducer` to finish request.
+
+NOTE: _Too low value_ of this timeout can lead to frequent failures of `IgniteToKafkaCdcStreamer`
Review Comment:
Good point!
##########
docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc:
##########
@@ -175,6 +183,16 @@ Kafka to Ignite configuration file should contain the following beans that will
| `threadCount` | Count of threads to proceed consumers. Each thread poll records from dedicated partitions in round-robin manner. | 16
|===
+`kafkaRequestTimeout` property sets timeouts for a following `KafkaConsumer` operations:
+
+- Poll from event and metadata topics.
+- Offsets commit.
+- Closing of a consumer.
+- Offsets request (during metadata updates).
+- Metadata topic partitions request (during metadata updater start).
+
+NOTE: _Too low value_ of `kafkaRequestTimeout` can lead to frequent failures of `kafka-to-ignite.sh` because of possible delays in network or on Kafka broker. Also, proper `KafkaConsumer` configuration should be provided, including link:https://kafka.apache.org/documentation/#consumerconfigs_request.timeout.ms[request.timeout.ms]. For more details you should refer to a link:https://kafka.apache.org/documentation/#configuration[configuration] section of the official Kafka documentation.
Review Comment:
Good point!
##########
docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc:
##########
@@ -175,6 +183,16 @@ Kafka to Ignite configuration file should contain the following beans that will
| `threadCount` | Count of threads to proceed consumers. Each thread poll records from dedicated partitions in round-robin manner. | 16
|===
+`kafkaRequestTimeout` property sets timeouts for a following `KafkaConsumer` operations:
+
+- Poll from event and metadata topics.
Review Comment:
Timeouts for `#poll` are, of cause, is on the surface. But I think, that we can at least tell about request types, that they are affected by a timeout, because they can (unlike `#poll`) raise timeout exceptions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] sonarcloud[bot] commented on pull request #10488: IGNITE-18574 CDC: add documentation about Kafka request timeouts
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #10488:
URL: https://github.com/apache/ignite/pull/10488#issuecomment-1501557880
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_ignite&pullRequest=10488)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10488&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10488&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_ignite&pullRequest=10488&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_ignite&pullRequest=10488&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10488&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_ignite&pullRequest=10488&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org