You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "wuzhenhua01 (via GitHub)" <gi...@apache.org> on 2023/03/18 03:51:00 UTC
[GitHub] [hudi] wuzhenhua01 opened a new pull request, #8227: [Hudi-5952] Fix NPE when use kafka callback
wuzhenhua01 opened a new pull request, #8227:
URL: https://github.com/apache/hudi/pull/8227
### Change Logs
Fix NPE when use kafka callback
### Impact
No impact
### Risk level (write none, low medium or high below)
none
### Documentation Update
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Change Logs and Impact were stated clearly
- [x] Adequate tests were added if applicable
- [x] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8227: [HUDI-5952] Fix NPE when use kafka callback
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8227:
URL: https://github.com/apache/hudi/pull/8227#issuecomment-1474678283
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8227: [HUDI-5952] Fix NPE when use kafka callback
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8227:
URL: https://github.com/apache/hudi/pull/8227#issuecomment-1474703766
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"triggerType" : "PUSH"
}, {
"hash" : "73861c38e3d67d687ef89ebff0fc08170892fec3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15780",
"triggerID" : "73861c38e3d67d687ef89ebff0fc08170892fec3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc UNKNOWN
* 73861c38e3d67d687ef89ebff0fc08170892fec3 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15780)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8227: [HUDI-5952] Fix NPE when use kafka callback
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8227:
URL: https://github.com/apache/hudi/pull/8227#issuecomment-1482412270
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"triggerType" : "PUSH"
}, {
"hash" : "73861c38e3d67d687ef89ebff0fc08170892fec3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15780",
"triggerID" : "73861c38e3d67d687ef89ebff0fc08170892fec3",
"triggerType" : "PUSH"
}, {
"hash" : "36abd1831338c963296e82e37d502db0deb5cc3b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15891",
"triggerID" : "36abd1831338c963296e82e37d502db0deb5cc3b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc UNKNOWN
* 73861c38e3d67d687ef89ebff0fc08170892fec3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15780)
* 36abd1831338c963296e82e37d502db0deb5cc3b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15891)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8227: [HUDI-5952] Fix NPE when use kafka callback
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8227:
URL: https://github.com/apache/hudi/pull/8227#issuecomment-1474749029
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"triggerType" : "PUSH"
}, {
"hash" : "73861c38e3d67d687ef89ebff0fc08170892fec3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15780",
"triggerID" : "73861c38e3d67d687ef89ebff0fc08170892fec3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc UNKNOWN
* 73861c38e3d67d687ef89ebff0fc08170892fec3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15780)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wuzhenhua01 commented on a diff in pull request #8227: [HUDI-5952] Fix NPE when use kafka callback
Posted by "wuzhenhua01 (via GitHub)" <gi...@apache.org>.
wuzhenhua01 commented on code in PR #8227:
URL: https://github.com/apache/hudi/pull/8227#discussion_r1142977268
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/callback/kafka/HoodieWriteCommitKafkaCallback.java:
##########
@@ -85,10 +85,10 @@ public KafkaProducer<String, String> createProducer(HoodieConfig hoodieConfig) {
kafkaProducerProps.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
// default "all" to ensure no message loss
kafkaProducerProps.setProperty(ProducerConfig.ACKS_CONFIG, hoodieConfig
- .getString(ACKS));
+ .getStringOrDefault(ACKS));
// retries 3 times by default
kafkaProducerProps.setProperty(ProducerConfig.RETRIES_CONFIG, hoodieConfig
- .getString(RETRIES));
+ .getStringOrDefault(RETRIES));
Review Comment:
yep
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 merged pull request #8227: [HUDI-5952] Fix NPE when use kafka callback
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #8227:
URL: https://github.com/apache/hudi/pull/8227
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] wuzhenhua01 commented on a diff in pull request #8227: [HUDI-5952] Fix NPE when use kafka callback
Posted by "wuzhenhua01 (via GitHub)" <gi...@apache.org>.
wuzhenhua01 commented on code in PR #8227:
URL: https://github.com/apache/hudi/pull/8227#discussion_r1147201398
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/callback/kafka/HoodieWriteCommitKafkaCallback.java:
##########
@@ -85,10 +85,10 @@ public KafkaProducer<String, String> createProducer(HoodieConfig hoodieConfig) {
kafkaProducerProps.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
// default "all" to ensure no message loss
kafkaProducerProps.setProperty(ProducerConfig.ACKS_CONFIG, hoodieConfig
- .getString(ACKS));
+ .getStringOrDefault(ACKS));
// retries 3 times by default
kafkaProducerProps.setProperty(ProducerConfig.RETRIES_CONFIG, hoodieConfig
- .getString(RETRIES));
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8227: [HUDI-5952] Fix NPE when use kafka callback
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8227:
URL: https://github.com/apache/hudi/pull/8227#issuecomment-1474698643
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"triggerType" : "PUSH"
}, {
"hash" : "73861c38e3d67d687ef89ebff0fc08170892fec3",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "73861c38e3d67d687ef89ebff0fc08170892fec3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc UNKNOWN
* 73861c38e3d67d687ef89ebff0fc08170892fec3 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #8227: [HUDI-5952] Fix NPE when use kafka callback
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #8227:
URL: https://github.com/apache/hudi/pull/8227#discussion_r1141566770
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/callback/kafka/HoodieWriteCommitKafkaCallback.java:
##########
@@ -85,10 +85,10 @@ public KafkaProducer<String, String> createProducer(HoodieConfig hoodieConfig) {
kafkaProducerProps.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
// default "all" to ensure no message loss
kafkaProducerProps.setProperty(ProducerConfig.ACKS_CONFIG, hoodieConfig
- .getString(ACKS));
+ .getStringOrDefault(ACKS));
// retries 3 times by default
kafkaProducerProps.setProperty(ProducerConfig.RETRIES_CONFIG, hoodieConfig
- .getString(RETRIES));
+ .getStringOrDefault(RETRIES));
Review Comment:
Does the default val work as expected?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8227: [HUDI-5952] Fix NPE when use kafka callback
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8227:
URL: https://github.com/apache/hudi/pull/8227#issuecomment-1482403734
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"triggerType" : "PUSH"
}, {
"hash" : "73861c38e3d67d687ef89ebff0fc08170892fec3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15780",
"triggerID" : "73861c38e3d67d687ef89ebff0fc08170892fec3",
"triggerType" : "PUSH"
}, {
"hash" : "36abd1831338c963296e82e37d502db0deb5cc3b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "36abd1831338c963296e82e37d502db0deb5cc3b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc UNKNOWN
* 73861c38e3d67d687ef89ebff0fc08170892fec3 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15780)
* 36abd1831338c963296e82e37d502db0deb5cc3b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8227: [HUDI-5952] Fix NPE when use kafka callback
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8227:
URL: https://github.com/apache/hudi/pull/8227#issuecomment-1482646921
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc",
"triggerType" : "PUSH"
}, {
"hash" : "73861c38e3d67d687ef89ebff0fc08170892fec3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15780",
"triggerID" : "73861c38e3d67d687ef89ebff0fc08170892fec3",
"triggerType" : "PUSH"
}, {
"hash" : "36abd1831338c963296e82e37d502db0deb5cc3b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15891",
"triggerID" : "36abd1831338c963296e82e37d502db0deb5cc3b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4cc7ab6ab87a640bcb68c97c55f642fde9ed5ecc UNKNOWN
* 36abd1831338c963296e82e37d502db0deb5cc3b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15891)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #8227: [HUDI-5952] Fix NPE when use kafka callback
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #8227:
URL: https://github.com/apache/hudi/pull/8227#discussion_r1143014743
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/callback/kafka/HoodieWriteCommitKafkaCallback.java:
##########
@@ -85,10 +85,10 @@ public KafkaProducer<String, String> createProducer(HoodieConfig hoodieConfig) {
kafkaProducerProps.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
// default "all" to ensure no message loss
kafkaProducerProps.setProperty(ProducerConfig.ACKS_CONFIG, hoodieConfig
- .getString(ACKS));
+ .getStringOrDefault(ACKS));
// retries 3 times by default
kafkaProducerProps.setProperty(ProducerConfig.RETRIES_CONFIG, hoodieConfig
- .getString(RETRIES));
Review Comment:
Is there any possibility we can write some test cases here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org