You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "weimingdiit (via GitHub)" <gi...@apache.org> on 2026/03/10 11:44:38 UTC
[PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
weimingdiit opened a new pull request, #9892:
URL: https://github.com/apache/ozone/pull/9892
## What changes were proposed in this pull request?
Avoid checking VersionedDatanodeFeatures.isFinalized(HBASE_SUPPORT) on every putBlock request. Cache the finalized state in BlockManagerImpl, and once finalization is observed, reuse the cached value for subsequent requests.
- Keep existing behavior before finalization: partial chunk list requests are rejected with UNSUPPORTED_REQUEST.
- Add a unit test to verify the finalization status is cached after it is first detected as finalized.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14591
## How was this patch tested?
unit tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on PR #9892:
URL: https://github.com/apache/ozone/pull/9892#issuecomment-4230630158
@smengcl @yandrey321
Not yet — this change was mainly motivated by the fact that putBlock is on the IO path and we currently do the feature-finalization check on every request.
That said, I agree this should be backed by profiling data rather than assumption. I’m planning to use a dedicated microbenchmark or a small reproducible test program for evaluation. What do you think?
I’ll collect a flame graph / profiler output for the putBlock workload and update the PR with:
- whether VersionedDatanodeFeatures.isFinalized (or the underlying version-manager check / lock path) shows up on the hot path
- before/after numbers
- the exact workload/setup used
If the profiling shows this is not a meaningful hotspot, I’m okay with dropping or reworking this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on PR #9892:
URL: https://github.com/apache/ozone/pull/9892#issuecomment-4033915094
@yandrey321 what is the motivation for this change? Was this causing a performance issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on PR #9892:
URL: https://github.com/apache/ozone/pull/9892#issuecomment-4034531357
> But in general IO path should not call feature checks outside of upgrade window unless these features could be altered in the runtime.
Finalization alters what features are available at runtime. Caching in general is a common source of bugs, and the version check is supposed to be so cheap it makes no difference. That is why I'm asking for numbers and not hypotheticals. Microbenchmarks risk adding complexity without real value.
I don't know if #9840 is going to change performance in any meaningful way, but it looks like a good change since it concentrates the concurrency handling to a central point instead of having a lock spread around to each method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
Posted by "yandrey321 (via GitHub)" <gi...@apache.org>.
yandrey321 commented on PR #9892:
URL: https://github.com/apache/ozone/pull/9892#issuecomment-4034459281
> #9840
this one is aimed to make feature check less expensive. But in general IO path should not call feature checks outside of upgrade window unless these features could be altered in the runtime.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9892:
URL: https://github.com/apache/ozone/pull/9892#issuecomment-4367539994
This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9892:
URL: https://github.com/apache/ozone/pull/9892#issuecomment-4627106087
Thank you for your contribution. This PR is being closed due to inactivity. Please contact a maintainer if you would like to reopen it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #9892: HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock
URL: https://github.com/apache/ozone/pull/9892
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
Posted by "yandrey321 (via GitHub)" <gi...@apache.org>.
yandrey321 commented on PR #9892:
URL: https://github.com/apache/ozone/pull/9892#issuecomment-4034825140
> > But in general IO path should not call feature checks outside of upgrade window unless these features could be altered in the runtime.
>
> Finalization alters what features are available at runtime. Caching in general is a common source of bugs, and the version check is supposed to be so cheap it makes no difference. That is why I'm asking for numbers and not hypotheticals. Microbenchmarks risk adding complexity without real value.
>
> I don't know if #9840 is going to change performance in any meaningful way, but it looks like a good change since it concentrates the concurrency handling to a central point instead of having a lock spread around to each method.
the idea is that we should avoid locks on the IO path. If we can improve Feature validation code to use atomic operations instead of concurrent maps/list it would be a huge win for code that check enabled/finalized features on IO path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on PR #9892:
URL: https://github.com/apache/ozone/pull/9892#issuecomment-4032299249
@smengcl @xichen01 @adoroszlai Hi, could you help review this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
Posted by "yandrey321 (via GitHub)" <gi...@apache.org>.
yandrey321 commented on PR #9892:
URL: https://github.com/apache/ozone/pull/9892#issuecomment-4033943302
> @yandrey321 what is the motivation for this change? Was this causing a performance issue?
Yes, its a performance issue, we check for feature finalization on every putBlock request instead of checking it only during upgrade/finalization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9892:
URL: https://github.com/apache/ozone/pull/9892#issuecomment-4166510511
This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14591. Cache HBASE_SUPPORT finalization status for putBlock [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on PR #9892:
URL: https://github.com/apache/ozone/pull/9892#issuecomment-4034385747
See also #9840
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org