You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "nilmadhab (via GitHub)" <gi...@apache.org> on 2025/02/13 08:25:10 UTC
[PR] Attempt to write a test case and fix left join bug [kafka]
nilmadhab opened a new pull request, #18887:
URL: https://github.com/apache/kafka/pull/18887
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*
*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [KAFKA-18713] Attempt to write a test case and fix left join bug [kafka]
Posted by "nilmadhab (via GitHub)" <gi...@apache.org>.
nilmadhab commented on PR #18887:
URL: https://github.com/apache/kafka/pull/18887#issuecomment-2657042434
<img width="1426" alt="Screenshot 2025-02-13 at 13 55 47" src="https://github.com/user-attachments/assets/5244a187-f1fd-4b7c-b2e2-9d8c79333351" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-18713: Fix left join bug by using DELETE_KEY_NO_PROPAGATE [kafka]
Posted by "nilmadhab (via GitHub)" <gi...@apache.org>.
nilmadhab commented on PR #18887:
URL: https://github.com/apache/kafka/pull/18887#issuecomment-2774464449
Yes, sure
Regards,
N. Mondal
+31687153557
Amsterdam, Netherlands
On Thu, 3 Apr 2025 at 05:49, Matthias J. Sax ***@***.***>
wrote:
> @nilmadhab <https://github.com/nilmadhab> -- I think we can close this PR
> in favor of the new PR you opened?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/kafka/pull/18887#issuecomment-2774410196>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABKO265R6XPASISO5NKPUA32XSVTFAVCNFSM6AAAAABXBRVO7SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONZUGQYTAMJZGY>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
> [image: mjsax]*mjsax* left a comment (apache/kafka#18887)
> <https://github.com/apache/kafka/pull/18887#issuecomment-2774410196>
>
> @nilmadhab <https://github.com/nilmadhab> -- I think we can close this PR
> in favor of the new PR you opened?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/kafka/pull/18887#issuecomment-2774410196>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABKO265R6XPASISO5NKPUA32XSVTFAVCNFSM6AAAAABXBRVO7SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDONZUGQYTAMJZGY>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [KAFKA-18713] Fix left join bug by using DELETE_KEY_NO_PROPAGATE [kafka]
Posted by "nilmadhab (via GitHub)" <gi...@apache.org>.
nilmadhab commented on PR #18887:
URL: https://github.com/apache/kafka/pull/18887#issuecomment-2664443563
https://issues.apache.org/jira/browse/KAFKA-18713
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-18713: Fix left join bug by using DELETE_KEY_NO_PROPAGATE [kafka]
Posted by "patronovski (via GitHub)" <gi...@apache.org>.
patronovski commented on PR #18887:
URL: https://github.com/apache/kafka/pull/18887#issuecomment-2694740863
Any updates on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-18713: Fix left join bug by using DELETE_KEY_NO_PROPAGATE [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on PR #18887:
URL: https://github.com/apache/kafka/pull/18887#issuecomment-2670317669
@nilmadhab -- I am actively working on my side to dig into FK join to refresh my memory and to fully understand the problem and fix, and will cycle back soon.
One thing I noticed is, that we actually should be able to use TTD to reproduce the issue. There is `KTableKTableForeignKeyJoinIntegrationTest` that we can re-use. The problem with `KTableKTableForeignKeyJoinIntegrationTest` is, that it uses `readKeyValuesToMap()` to verify the result. For this case, if we get two results for the same key, only the last results is checked (as any previous result is overwritten when we produce the result `Map`)... In general, we should never get more than one result for each update to the left hand side table, but if it happens the test does not catch it.
While we might want to refactor the test and use `readKeyValuesToList()` instead, it might be something we should not piggy-back on this PR, but do an independent PR to update the test. However, when you add a new test for the fix, you could use `readKeyValuesToList()` and it should simplify testing significantly, instead of the current approach to use an "heavy weight" integration test for it. Thoughts?
Btw, there is also an older PR https://github.com/apache/kafka/pull/15607 which fixes something similar, that I try to re-activate. We need to coordinate both PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [KAFKA-18713] Attempt to write a test case and fix left join bug [kafka]
Posted by "nilmadhab (via GitHub)" <gi...@apache.org>.
nilmadhab commented on PR #18887:
URL: https://github.com/apache/kafka/pull/18887#issuecomment-2657040824
![Uploading Screenshot 2025-02-13 at 13.55.47.png…]()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-18713: Fix left join bug by using DELETE_KEY_NO_PROPAGATE [kafka]
Posted by "nilmadhab (via GitHub)" <gi...@apache.org>.
nilmadhab commented on PR #18887:
URL: https://github.com/apache/kafka/pull/18887#issuecomment-2694913447
> Any updates on this?
https://github.com/apache/kafka/pull/19005
I have created a new PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Attempt to write a test case and fix left join bug [kafka]
Posted by "nilmadhab (via GitHub)" <gi...@apache.org>.
nilmadhab commented on PR #18887:
URL: https://github.com/apache/kafka/pull/18887#issuecomment-2655855496
![Uploading Screenshot 2025-02-13 at 13.55.47.png…]()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-18713: Fix left join bug by using DELETE_KEY_NO_PROPAGATE [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on PR #18887:
URL: https://github.com/apache/kafka/pull/18887#issuecomment-2774410196
@nilmadhab -- I think we can close this PR in favor of the new PR you opened?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-18713: Fix left join bug by using DELETE_KEY_NO_PROPAGATE [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax closed pull request #18887: KAFKA-18713: Fix left join bug by using DELETE_KEY_NO_PROPAGATE
URL: https://github.com/apache/kafka/pull/18887
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-18713: Fix left join bug by using DELETE_KEY_NO_PROPAGATE [kafka]
Posted by "nilmadhab (via GitHub)" <gi...@apache.org>.
nilmadhab commented on PR #18887:
URL: https://github.com/apache/kafka/pull/18887#issuecomment-2670380898
@mjsax it makes sense to me.
1. I can remove the` KTableKTableForeignKeyLeftJoinDistributedTest.java` in this MR.
2. I will check if I can reproduce the issue `KTableKTableForeignKeyJoinIntegrationTest` by using` readKeyValuesToList()` and create a separate MR for that, so we can skip "heavy weight" integration test, as it will make testing more efficient.
3. I will go through the [PR](https://github.com/apache/kafka/pull/15607)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org