You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "YutaLin (via GitHub)" <gi...@apache.org> on 2026/05/02 02:38:40 UTC
[PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
YutaLin opened a new pull request, #10170:
URL: https://github.com/apache/ozone/pull/10170
## What changes were proposed in this pull request?
Use a separate OpenKeyInfo for openKeyTable to differentiate with the final KeyInfo in the keyTable
Please describe your PR in detail:
Base on discussion: https://github.com/apache/ozone/pull/9815#discussion_r3013645782
We've add `expectedDataGeneration` and `expectedETag` in KeyInfo but these value should not be stored after commit, otherwise it has space overhead, so we introduced `OpenKeyInfo` to prevent space waste.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14957
## How was this patch tested?
Update tests with `OmOpenKeyInfo` and run CI Action(https://github.com/YutaLin/ozone/actions/runs/25239339785)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "peterxcli (via GitHub)" <gi...@apache.org>.
peterxcli commented on PR #10170:
URL: https://github.com/apache/ozone/pull/10170#issuecomment-4364371814
I was thinking if we could just introduce a open key info serde class that only store a subset of key info, without, no explicitly unassign need, only need to build the data class from the incoming key info in request.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "YutaLin (via GitHub)" <gi...@apache.org>.
YutaLin commented on PR #10170:
URL: https://github.com/apache/ozone/pull/10170#issuecomment-4374519465
> I don't think this is a compatible change since we are changing the DB definition in-place.
Hi @ivandika3, can you elaborate?
Would it be acceptable to implement a backward compatible codec that first attempts to serialize OpenKeyInfo and falls back to KeyInfo if that attempt fails?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on PR #10170:
URL: https://github.com/apache/ozone/pull/10170#issuecomment-4362723792
Thanks @YutaLin for the patch, but I don't think this is a compatible change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "YutaLin (via GitHub)" <gi...@apache.org>.
YutaLin commented on PR #10170:
URL: https://github.com/apache/ozone/pull/10170#issuecomment-4364340726
Hey @ivandika3,
> adding fields to protobuf would not cause increased protobuf size as long as we remember to unassign the fields.
Thanks for review, I will close this patch since it's no longer needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "YutaLin (via GitHub)" <gi...@apache.org>.
YutaLin commented on PR #10170:
URL: https://github.com/apache/ozone/pull/10170#issuecomment-4501458817
Hi @szetszwo
It would be great if you could help take a look at this. We have different perspectives on the codec implementation and its impact. Appreciate your input!
This is related to #10181
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "YutaLin (via GitHub)" <gi...@apache.org>.
YutaLin closed pull request #10170: HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo
URL: https://github.com/apache/ozone/pull/10170
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on PR #10170:
URL: https://github.com/apache/ozone/pull/10170#issuecomment-4521049030
@YutaLin , thanks for pinging me. Will review https://github.com/apache/ozone/pull/10181
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "YutaLin (via GitHub)" <gi...@apache.org>.
YutaLin commented on PR #10170:
URL: https://github.com/apache/ozone/pull/10170#issuecomment-4362700017
Hi @peterxcli, @ivandika3
Please help me review this, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on PR #10170:
URL: https://github.com/apache/ozone/pull/10170#issuecomment-4364079917
IMO it is fine to leave things be unless there is a good reason other than separation of concern (e.g. https://issues.apache.org/jira/browse/HDDS-10611).
From the overhead point of view, adding fields to protobuf would not cause protobuf size to increase as long as we remember to unassign the fields.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on PR #10170:
URL: https://github.com/apache/ozone/pull/10170#issuecomment-4375846720
@YutaLin The new OpenKeyInfo proto and KeyInfo proto are not compatible. For example, the field 1 for `OpenKeyInfo` is a `KeyInfo`, but field 1 for `KeyInfo` is volumeName. This means it's not backward and forward compatible.
> Would it be acceptable to implement a backward compatible codec that first attempts to serialize OpenKeyInfo and falls back to KeyInfo if that attempt fails?
No I don't think so, duplicate codec serde might cause performance issues since openKeyTable & openFileTable is in the critical path. I think the approach in https://github.com/apache/ozone/pull/10181 might be enough.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "peterxcli (via GitHub)" <gi...@apache.org>.
peterxcli commented on PR #10170:
URL: https://github.com/apache/ozone/pull/10170#issuecomment-4377552551
> The new OpenKeyInfo proto and KeyInfo proto are not compatible.
How about we add a `Codec` for each OM definition, where each `Codec` is actually a projection of a subset of the `KeyInfo` proto’s properties.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "YutaLin (via GitHub)" <gi...@apache.org>.
YutaLin commented on PR #10170:
URL: https://github.com/apache/ozone/pull/10170#issuecomment-4365452917
Hi @peterxcli, that's a great feedback, I will raise another patch. thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14957. Separate OpenKeyInfo to differentiate KeyInfo [ozone]
Posted by "peterxcli (via GitHub)" <gi...@apache.org>.
peterxcli commented on PR #10170:
URL: https://github.com/apache/ozone/pull/10170#issuecomment-4364373355
if you agree with my point, let's reopen this and refactor it! thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org