You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/06/30 18:22:36 UTC
[GitHub] [druid] kfaraz opened a new pull request, #12722: Fix expiry timeout bug in LocalIntermediateDataManager
kfaraz opened a new pull request, #12722:
URL: https://github.com/apache/druid/pull/12722
### Description
Fixes one of the bugs reported in #12692
As observed in [this comment](https://github.com/apache/druid/issues/12692#issuecomment-1166225796),
the expiry timeout is compared against the current time but the condition is reversed.
This means that as soon as a supervisor finishes, it gets cleaned up, irrespective of the specified
`intermediaryPartitionTimeout` period.
### Changes
* Fix the condition
* Add tests to verify the new correct behaviour
* Reduce the default expiry timeout from P1D to PT5M
to retain current behaviour in case of default configs.
### Upgrade effects
On upgrade,
1. Clusters that have overridden `intermediaryPartitionCleanupPeriodSec`
but not `intermediaryPartitionTimeout` might notice intermediate data
from completed supervisors lingering for up to 5 more minutes.
2. Clusters that have overridden `intermediaryPartitionTimeout` but not
`intermediaryPartitionCleanupPeriodSec` might notice intermediate data
lingering for up to the specified partition timeout duration (i.e. their partition
timeout will now start getting honored).
3. Clusters that have not overridden these configs will not notice any change.
<hr>
This PR has:
- [ ] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [ ] added integration tests.
- [ ] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] kfaraz commented on pull request #12722: Fix expiry timeout bug in LocalIntermediateDataManager
Posted by GitBox <gi...@apache.org>.
kfaraz commented on PR #12722:
URL: https://github.com/apache/druid/pull/12722#issuecomment-1172220778
Thanks for the review, @clintropolis !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] kfaraz merged pull request #12722: Fix expiry timeout bug in LocalIntermediateDataManager
Posted by GitBox <gi...@apache.org>.
kfaraz merged PR #12722:
URL: https://github.com/apache/druid/pull/12722
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org