You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "sumitagrawl (via GitHub)" <gi...@apache.org> on 2025/06/17 06:57:37 UTC
[PR] HDDS-13286. write fail for volume full during stream write [ozone]
sumitagrawl opened a new pull request, #8644:
URL: https://github.com/apache/ozone/pull/8644
## What changes were proposed in this pull request?
When volume if full, means no space available for data write, need to make write failure as Disk out of space.
This needs to be done for write operation during stream write.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-13286
## How was this patch tested?
- Unit test is added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2155297924
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java:
##########
@@ -93,6 +96,17 @@ public final boolean isOpen() {
return getChannel().isOpen();
}
+ protected void assertSpaceAvailability(int remaining) throws IOException {
+ HddsVolume volume = containerData.getVolume();
+ SpaceUsageSource currentUsage = volume.getCurrentUsage();
+ if (currentUsage.getAvailable() - volume.getFreeSpaceToSpare(currentUsage.getCapacity()) <= remaining) {
+ throw new StorageContainerException("write failed for container " + containerData.getContainerID()
+ + " with size " + remaining + " due to volume " + volume.getStorageID() + " out of space "
+ + volume.getCurrentUsage() + " with minimum free space required: "
+ + volume.getFreeSpaceToSpare(currentUsage.getCapacity()), DISK_OUT_OF_SPACE);
+ }
+ }
Review Comment:
- Please rename `remaining` to `requested`.
- Do not call `getCurrentUsage()`/`getFreeSpaceToSpare(..)` twice -- two values could be different.
- Throws StorageContainerException
- Revise the error message as below:
```java
protected void assertSpaceAvailability(int requested) throws StorageContainerException {
final HddsVolume volume = containerData.getVolume();
final SpaceUsageSource currentUsage = volume.getCurrentUsage();
final long spared = volume.getFreeSpaceToSpare(currentUsage.getCapacity());
if (currentUsage.getAvailable() - spared <= requested) {
throw new StorageContainerException("Failed to write " + requested + " bytes to container "
+ containerData.getContainerID() + " due to volume " + volume.getStorageID() + " out of space "
+ currentUsage + ", spared=" + spared, DISK_OUT_OF_SPACE);
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2153737324
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java:
##########
@@ -93,6 +94,14 @@ public final boolean isOpen() {
return getChannel().isOpen();
}
+ protected void assertSpaceAvailability() throws IOException {
+ if (containerData.getVolume().isVolumeFull()) {
+ throw new StorageContainerException("write failed for container " + containerData.getContainerID()
+ + " due to volume " + containerData.getVolume().getStorageID() + " out of space "
+ + containerData.getVolume().getCurrentUsage(), DISK_OUT_OF_SPACE);
Review Comment:
- Please move method to `HddsVolume` as `assertSpaceForWrite`.
- Accept required size in parameter, as @szetszwo [mentioned](https://github.com/apache/ozone/pull/8644/files#r2153308528).
- In the implementation, add local variables for the values (`currentUsage`, `minFreeSpace`), and use them in both calculation and log message. Do not call `boolean isVolumeFull()`, because then the message needs to obtain the same values again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "sumitagrawl (via GitHub)" <gi...@apache.org>.
sumitagrawl commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2158029365
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java:
##########
@@ -341,4 +344,16 @@ public static long getPendingDeletionBlocks(ContainerData containerData) {
" not support.");
}
}
+
+ public static void assertSpaceAvailability(long containerId, HddsVolume volume, int sizeRequested)
+ throws StorageContainerException {
+ final SpaceUsageSource currentUsage = volume.getCurrentUsage();
+ final long spared = volume.getFreeSpaceToSpare(currentUsage.getCapacity());
+
+ if (currentUsage.getAvailable() - spared <= sizeRequested) {
Review Comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo merged PR #8644:
URL: https://github.com/apache/ozone/pull/8644
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2155454472
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java:
##########
@@ -93,6 +96,17 @@ public final boolean isOpen() {
return getChannel().isOpen();
}
+ protected void assertSpaceAvailability(int remaining) throws IOException {
+ HddsVolume volume = containerData.getVolume();
+ SpaceUsageSource currentUsage = volume.getCurrentUsage();
+ if (currentUsage.getAvailable() - volume.getFreeSpaceToSpare(currentUsage.getCapacity()) <= remaining) {
+ throw new StorageContainerException("write failed for container " + containerData.getContainerID()
+ + " with size " + remaining + " due to volume " + volume.getStorageID() + " out of space "
+ + volume.getCurrentUsage() + " with minimum free space required: "
+ + volume.getFreeSpaceToSpare(currentUsage.getCapacity()), DISK_OUT_OF_SPACE);
+ }
+ }
Review Comment:
Note: `spare(d)` is called `min.free.space` in the config, so I think it's better to reflect that in the code/message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2157532866
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java:
##########
@@ -341,4 +344,16 @@ public static long getPendingDeletionBlocks(ContainerData containerData) {
" not support.");
}
}
+
+ public static void assertSpaceAvailability(long containerId, HddsVolume volume, int sizeRequested)
+ throws StorageContainerException {
+ final SpaceUsageSource currentUsage = volume.getCurrentUsage();
+ final long spared = volume.getFreeSpaceToSpare(currentUsage.getCapacity());
+
+ if (currentUsage.getAvailable() - spared <= sizeRequested) {
Review Comment:
@sumitagrawl , It should be `<` since `==` means that the space is enough. Sorry that I did not see it earlier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2151474058
##########
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/impl/TestKeyValueStreamDataChannel.java:
##########
@@ -151,6 +159,25 @@ private static ContainerCommandRequestProto readPutBlockRequest(ByteBuffer b)
return request;
}
+ @Test
+ public void testVolumeFullCase() throws Exception {
+ File tempFile = File.createTempFile("test-kv-stream", ".tmp");
+ HddsVolume mockVolume = mock(HddsVolume.class);
+ when(mockVolume.getStorageID()).thenReturn("storageId");
Review Comment:
Consider scheduling deletion of the temporary file (e.g., using tempFile.deleteOnExit()) to ensure proper cleanup after tests.
```suggestion
tempFile.deleteOnExit();
HddsVolume mockVolume = mock(HddsVolume.class);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2154327453
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java:
##########
@@ -93,6 +94,14 @@ public final boolean isOpen() {
return getChannel().isOpen();
}
+ protected void assertSpaceAvailability() throws IOException {
+ if (containerData.getVolume().isVolumeFull()) {
+ throw new StorageContainerException("write failed for container " + containerData.getContainerID()
+ + " due to volume " + containerData.getVolume().getStorageID() + " out of space "
+ + containerData.getVolume().getCurrentUsage(), DISK_OUT_OF_SPACE);
Review Comment:
Also, please rename to `assertSpaceForWrite` for clarity and simplicity, and rename parameter `remaining` (which is in terms of the buffer) to `required`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2156171577
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java:
##########
@@ -93,6 +96,18 @@ public final boolean isOpen() {
return getChannel().isOpen();
}
+ protected void assertSpaceAvailability(int requested) throws StorageContainerException {
+ final HddsVolume volume = containerData.getVolume();
+ final SpaceUsageSource currentUsage = volume.getCurrentUsage();
+ final long spared = volume.getFreeSpaceToSpare(currentUsage.getCapacity());
+
+ if (currentUsage.getAvailable() - spared <= requested) {
+ throw new StorageContainerException("Failed to write " + requested + " bytes to container "
+ + containerData.getContainerID() + " due to volume " + volume.getStorageID() + " out of space "
+ + currentUsage + ", spared=" + spared, DISK_OUT_OF_SPACE);
+ }
+ }
Review Comment:
If you don't want to accept my [suggestion](https://github.com/apache/ozone/pull/8644#discussion_r2154304362) about moving this to `HddsVolume` with passing the container-specific part of the message as a parameter, then at least please move it to `ContainerUtils`. It should be reused for non-streaming write in #8642.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "sumitagrawl (via GitHub)" <gi...@apache.org>.
sumitagrawl commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2156969635
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java:
##########
@@ -93,6 +96,17 @@ public final boolean isOpen() {
return getChannel().isOpen();
}
+ protected void assertSpaceAvailability(int remaining) throws IOException {
+ HddsVolume volume = containerData.getVolume();
+ SpaceUsageSource currentUsage = volume.getCurrentUsage();
+ if (currentUsage.getAvailable() - volume.getFreeSpaceToSpare(currentUsage.getCapacity()) <= remaining) {
+ throw new StorageContainerException("write failed for container " + containerData.getContainerID()
+ + " with size " + remaining + " due to volume " + volume.getStorageID() + " out of space "
+ + volume.getCurrentUsage() + " with minimum free space required: "
+ + volume.getFreeSpaceToSpare(currentUsage.getCapacity()), DISK_OUT_OF_SPACE);
+ }
+ }
Review Comment:
@adoroszlai updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "sumitagrawl (via GitHub)" <gi...@apache.org>.
sumitagrawl commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2156970186
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java:
##########
@@ -93,6 +96,18 @@ public final boolean isOpen() {
return getChannel().isOpen();
}
+ protected void assertSpaceAvailability(int requested) throws StorageContainerException {
+ final HddsVolume volume = containerData.getVolume();
+ final SpaceUsageSource currentUsage = volume.getCurrentUsage();
+ final long spared = volume.getFreeSpaceToSpare(currentUsage.getCapacity());
+
+ if (currentUsage.getAvailable() - spared <= requested) {
+ throw new StorageContainerException("Failed to write " + requested + " bytes to container "
+ + containerData.getContainerID() + " due to volume " + volume.getStorageID() + " out of space "
+ + currentUsage + ", spared=" + spared, DISK_OUT_OF_SPACE);
+ }
+ }
Review Comment:
ok, moved to the ContainerUtils
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. Fail stream write when the volume is full. [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on PR #8644:
URL: https://github.com/apache/ozone/pull/8644#issuecomment-2992392485
@sumitagrawl , thanks for working on this!
@adoroszlai , thanks for reviewing this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "sumitagrawl (via GitHub)" <gi...@apache.org>.
sumitagrawl commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2154005557
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java:
##########
@@ -93,6 +94,14 @@ public final boolean isOpen() {
return getChannel().isOpen();
}
+ protected void assertSpaceAvailability() throws IOException {
+ if (containerData.getVolume().isVolumeFull()) {
+ throw new StorageContainerException("write failed for container " + containerData.getContainerID()
+ + " due to volume " + containerData.getVolume().getStorageID() + " out of space "
+ + containerData.getVolume().getCurrentUsage(), DISK_OUT_OF_SPACE);
Review Comment:
done
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/KeyValueStreamDataChannel.java:
##########
@@ -59,6 +59,7 @@ public int write(ReferenceCountedObject<ByteBuffer> referenceCounted)
throws IOException {
getMetrics().incContainerOpsMetrics(getType());
assertOpen();
+ assertSpaceAvailability();
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "sumitagrawl (via GitHub)" <gi...@apache.org>.
sumitagrawl commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2154041320
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java:
##########
@@ -93,6 +94,14 @@ public final boolean isOpen() {
return getChannel().isOpen();
}
+ protected void assertSpaceAvailability() throws IOException {
+ if (containerData.getVolume().isVolumeFull()) {
+ throw new StorageContainerException("write failed for container " + containerData.getContainerID()
+ + " due to volume " + containerData.getVolume().getStorageID() + " out of space "
+ + containerData.getVolume().getCurrentUsage(), DISK_OUT_OF_SPACE);
Review Comment:
@adoroszlai others are done, but unable to move to HddsVolume as log message needs `containerId` which is not present as part of HddsVolume. For log completeness, it should be done at outer layer, or need encapsulate exception again adding more details at outer layer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2157532866
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/helpers/ContainerUtils.java:
##########
@@ -341,4 +344,16 @@ public static long getPendingDeletionBlocks(ContainerData containerData) {
" not support.");
}
}
+
+ public static void assertSpaceAvailability(long containerId, HddsVolume volume, int sizeRequested)
+ throws StorageContainerException {
+ final SpaceUsageSource currentUsage = volume.getCurrentUsage();
+ final long spared = volume.getFreeSpaceToSpare(currentUsage.getCapacity());
+
+ if (currentUsage.getAvailable() - spared <= sizeRequested) {
Review Comment:
@sumitagrawl , It should be `<` since `==` means that the space is enough.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #8644:
URL: https://github.com/apache/ozone/pull/8644#issuecomment-2986742457
On a related note: I don't think it's good practice to have two or more pull requests in parallel with similar code change handling different case (stream vs. non-stream write path).
You inevitably get duplication in one or more areas:
- review effort
- your effort (e.g. to tweak the exception message in #8642 based on comment made here)
- code
- CI
Depending on the size of the changes, please either:
- create one PR at a time, wait for it to be merged, then your next PR starts in a better shape, OR
- if the patches are small and changes are related, submit a single combined PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2154304362
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java:
##########
@@ -93,6 +94,14 @@ public final boolean isOpen() {
return getChannel().isOpen();
}
+ protected void assertSpaceAvailability() throws IOException {
+ if (containerData.getVolume().isVolumeFull()) {
+ throw new StorageContainerException("write failed for container " + containerData.getContainerID()
+ + " due to volume " + containerData.getVolume().getStorageID() + " out of space "
+ + containerData.getVolume().getCurrentUsage(), DISK_OUT_OF_SPACE);
Review Comment:
Pass container-specific part of the message as `Supplier<String>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-13286. write fail for volume full during stream write [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #8644:
URL: https://github.com/apache/ozone/pull/8644#discussion_r2153308528
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/KeyValueStreamDataChannel.java:
##########
@@ -59,6 +59,7 @@ public int write(ReferenceCountedObject<ByteBuffer> referenceCounted)
throws IOException {
getMetrics().incContainerOpsMetrics(getType());
assertOpen();
+ assertSpaceAvailability();
Review Comment:
It should account for `referenceCounted.get().remaining()`.
##########
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/impl/StreamDataChannelBase.java:
##########
@@ -93,6 +94,14 @@ public final boolean isOpen() {
return getChannel().isOpen();
}
+ protected void assertSpaceAvailability() throws IOException {
+ if (containerData.getVolume().isVolumeFull()) {
+ throw new StorageContainerException("write failed for container " + containerData.getContainerID()
+ + " due to volume " + containerData.getVolume().getStorageID() + " out of space "
+ + containerData.getVolume().getCurrentUsage(), DISK_OUT_OF_SPACE);
Review Comment:
We should print also the reserved space and the required space; otherwise, the message will be confusing since it will show space available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org