You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2021/08/10 10:18:14 UTC
[GitHub] [ozone] sadanand48 opened a new pull request #2520: HDDS-5598. Validate block file length during the first writeChunk
sadanand48 opened a new pull request #2520:
URL: https://github.com/apache/ozone/pull/2520
## What changes were proposed in this pull request?
Add a defensive check to ensure that block file is empty during the first write chunk at 0 offset.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5598
## How was this patch tested?
CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sadanand48 closed pull request #2520: HDDS-5598. Validate block file length during writeChunk
Posted by GitBox <gi...@apache.org>.
sadanand48 closed pull request #2520:
URL: https://github.com/apache/ozone/pull/2520
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] mukul1987 edited a comment on pull request #2520: HDDS-5598. Validate block file length during the first writeChunk
Posted by GitBox <gi...@apache.org>.
mukul1987 edited a comment on pull request #2520:
URL: https://github.com/apache/ozone/pull/2520#issuecomment-896408659
Agree to comment from @adoroszlai
public static boolean validateChunkForOverwrite(File chunkFile,
ChunkInfo info) {
if (isOverWriteRequested(chunkFile, info)) {
if (!isOverWritePermitted(info)) {
LOG.warn("Duplicate write chunk request. Chunk overwrite " +
"without explicit request. {}", info);
}
return true;
}
return false;
}
Apart from logging a warning can we also change it to a precondition to fail the write ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sadanand48 commented on pull request #2520: HDDS-5598. Validate block file length during the first writeChunk
Posted by GitBox <gi...@apache.org>.
sadanand48 commented on pull request #2520:
URL: https://github.com/apache/ozone/pull/2520#issuecomment-897013106
> Apart from logging a warning can we also change it to a precondition to fail the write ?
In the current version of the patch , this check would happen only on enabling a config which I have disabled by default. If the config is turned on , the write would fail if offset!=length and there are certain scenarios where this can cause an issue observed by unit tests. For example if a datanode receives a writechunk after a container dir is deleted, it would fail the writechunk without marking container unhealthy and the container would still be open (TestContainerStateMachine#testContainerStateMachineFailure) . Need to handle such cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] avijayanhwx commented on pull request #2520: HDDS-5598. Validate block file length during the first writeChunk
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on pull request #2520:
URL: https://github.com/apache/ozone/pull/2520#issuecomment-896258331
@sadanand48 Is this patch complete? I see a patch file attached to the JIRA that has more changes.
We can have a duplicate write chunk 1 if the ratis txn failed or timed out. I believe we cannot fail the transaction for just this alone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] sadanand48 commented on pull request #2520: HDDS-5598. Validate block file length during writeChunk
Posted by GitBox <gi...@apache.org>.
sadanand48 commented on pull request #2520:
URL: https://github.com/apache/ozone/pull/2520#issuecomment-903626560
Closing this as chunk validations are added in PR https://github.com/apache/ozone/pull/2542
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] mukul1987 commented on pull request #2520: HDDS-5598. Validate block file length during the first writeChunk
Posted by GitBox <gi...@apache.org>.
mukul1987 commented on pull request #2520:
URL: https://github.com/apache/ozone/pull/2520#issuecomment-896408659
Agree to comment from @adoroszlai
` public static boolean validateChunkForOverwrite(File chunkFile,
ChunkInfo info) {
if (isOverWriteRequested(chunkFile, info)) {
if (!isOverWritePermitted(info)) {
LOG.warn("Duplicate write chunk request. Chunk overwrite " +
"without explicit request. {}", info);
}
return true;
}
return false;
}
`
Apart from logging a warning can we also change it to a precondition to fail the write ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] mukul1987 edited a comment on pull request #2520: HDDS-5598. Validate block file length during the first writeChunk
Posted by GitBox <gi...@apache.org>.
mukul1987 edited a comment on pull request #2520:
URL: https://github.com/apache/ozone/pull/2520#issuecomment-896408659
Agree to comment from @adoroszlai
public static boolean validateChunkForOverwrite(File chunkFile,
ChunkInfo info) {
if (isOverWriteRequested(chunkFile, info)) {
if (!isOverWritePermitted(info)) {
LOG.warn("Duplicate write chunk request. Chunk overwrite " +
"without explicit request. {}", info);
}
return true;
}
return false;
}
Apart from logging a warning can we also change it to a precondition to fail the write ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org