You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/03/02 09:00:01 UTC
[GitHub] [ozone] kaijchen opened a new pull request #3151: HDDS-6400. EC: Refactor ECKeyOutputStream
kaijchen opened a new pull request #3151:
URL: https://github.com/apache/ozone/pull/3151
## What changes were proposed in this pull request?
Refactor ECKeyOutputStream to make the code cleaner.
Reuse code when writing the last stripe or rewriting failed stripes.
The changes includes:
1. Remove isFullCell argument in handleDataWrite()
2. Move checkAndWriteParityCells() inside handleDataWrite()
3. Change checkAndWriteParityCells() to encodeAndWriteParityCells()
4. Support partial stripe in encodeAndWriteParityCells()
5. Update acked position in handleParityWrites()
6. Seperate encode from handleParityWrites()
7. Reuse handleParityWrites() in rewriteStripeToNewBlockGroup()
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-6400
## How was this patch tested?
CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen closed pull request #3151: HDDS-6400. EC: [Refactor] Reuse code to write parity in close and retry
Posted by GitBox <gi...@apache.org>.
kaijchen closed pull request #3151:
URL: https://github.com/apache/ozone/pull/3151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on pull request #3151: HDDS-6400. EC: Refactor ECKeyOutputStream
Posted by GitBox <gi...@apache.org>.
kaijchen commented on pull request #3151:
URL: https://github.com/apache/ozone/pull/3151#issuecomment-1057053603
I have squashed the commits to make the review easier. (It's easier to review it step by step)
P.S. Should we delay data cell writes until the stripe is ready?
(To improve code reuse in the retry path)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on pull request #3151: HDDS-6400. EC: Refactor ECKeyOutputStream
Posted by GitBox <gi...@apache.org>.
sodonnel commented on pull request #3151:
URL: https://github.com/apache/ozone/pull/3151#issuecomment-1057081322
Is this refactor adding any features or fixing any bugs? As it stands we are still trying to stabilise the code - these sort of changes to make it "cleaner" take time to review and risk adding more problems, especially when the test suite is not covering all possible scenarios.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on pull request #3151: HDDS-6400. EC: [Refactor] Reuse code to write parity in close and retry
Posted by GitBox <gi...@apache.org>.
kaijchen commented on pull request #3151:
URL: https://github.com/apache/ozone/pull/3151#issuecomment-1069040287
Closed by breaking down this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kaijchen commented on pull request #3151: HDDS-6400. EC: Refactor ECKeyOutputStream
Posted by GitBox <gi...@apache.org>.
kaijchen commented on pull request #3151:
URL: https://github.com/apache/ozone/pull/3151#issuecomment-1057097235
> Is this refactor adding any features or fixing any bugs? As it stands we are still trying to stabilise the code - these sort of changes to make it "cleaner" take time to review and risk adding more problems, especially when the test suite is not covering all possible scenarios.
No, this PR does not suppose to change code logic. I agree with your point. We can revisit this later if there is a need.
For example, we are going to implement cell rotation feature and minimum required write feature later. It will be easier to change fewer places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sodonnel commented on pull request #3151: HDDS-6400. EC: Reuse code to write parity in close and retry
Posted by GitBox <gi...@apache.org>.
sodonnel commented on pull request #3151:
URL: https://github.com/apache/ozone/pull/3151#issuecomment-1057168053
Lets see what @umamaheswararao thinks of the changes here. I am not against improvements, but we need to get the feature stabilised and refactoring things may not be helping with that. We could also do with improving unit / integration test coverage and scenarios.
> we are going to implement cell rotation feature
We are not planning to do that for now - the design needs updated. It just adds more complexity and in real world use, its hard to see it it will give us much, unless we are storing only tiny keys, where EC is not the correct use case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org