You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "chitralverma (via GitHub)" <gi...@apache.org> on 2025/05/30 06:00:49 UTC
[I] bug: (services/azblob) `if_not_exist` capability is not working on write [opendal]
chitralverma opened a new issue, #6235:
URL: https://github.com/apache/opendal/issues/6235
### Describe the bug
`write_with_if_not_exists` capability is enabled on azblob service but it seems that when an object is written with `if_not_exists` option, the operation breaks
Ref: https://github.com/apache/opendal/actions/runs/15340065425/job/43164417657?pr=6086
### Steps to Reproduce
```py
@pytest.mark.need_capability("write", "delete", "write_with_if_not_exists")
def test_sync_writer_options(service_name, operator, async_operator):
size = randint(1, 1024)
filename = f"test_file_{str(uuid4())}.txt"
content = os.urandom(size)
f = operator.open(filename, "wb")
written_bytes = f.write(content)
assert written_bytes == size
f.close()
with pytest.raises(ConditionNotMatch):
with operator.open(filename, "wb", if_not_exists=True) as w:
w.write(content)
```
### Expected Behavior
if the file exists error should be thrown as expected.
### Additional Context
_No response_
### Are you willing to submit a PR to fix this bug?
- [ ] Yes, I would like to submit a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: (services/azblob) `if_not_exist` capability is not working on write [opendal]
Posted by "chitralverma (via GitHub)" <gi...@apache.org>.
chitralverma commented on issue #6235:
URL: https://github.com/apache/opendal/issues/6235#issuecomment-2921725659
yes seems like it, maybe it's an inconsistency issue? because other services throw condition not match error
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug(bindings/python): Python bindings can throw different error types for the same error code [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo closed issue #6235: bug(bindings/python): Python bindings can throw different error types for the same error code
URL: https://github.com/apache/opendal/issues/6235
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: (services/azblob) `if_not_exist` capability is not working on write [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #6235:
URL: https://github.com/apache/opendal/issues/6235#issuecomment-2921603821
We already have this feature implemented in Rust core, so the error should already be thrown. One possible issue here is that it's no longer `ConditionNotMatch`.
https://github.com/apache/opendal/blob/60f4a0d5c5fcd74bbd2ee52c3e35f9d73c9af0ea/bindings/python/src/file.rs#L511-L523
Here, we are converting OpenDAL's error to `PyIOError`, so it's no longer the original OpenDAL error.
One possible workaround is to check the error message, but I'm not sure if that's a good idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: (services/azblob) `if_not_exist` capability is not working on write [opendal]
Posted by "chitralverma (via GitHub)" <gi...@apache.org>.
chitralverma commented on issue #6235:
URL: https://github.com/apache/opendal/issues/6235#issuecomment-2922081045
Got it.
so I can see that `write_with_if_not_exists` is True for local fs, azblob, ghac, azdls, gcs and s3 services. out of these, there are no tests for ghac, azdls, gcs. and failure was reported for azblob and s3 (minio).
Inconsistency part: For some reason the same issue was not thrown for local fs even though the tests ran.
If a user creates an operator with the `if_not_exists` option on write, either the operation should fail eagerly if the operator does not have this capability or it should work as expected, throwing the same type of error across services.
How do you suggest, this to be handled, i can include this in the existing MR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: (services/azblob) `if_not_exist` capability is not working on write [opendal]
Posted by "chitralverma (via GitHub)" <gi...@apache.org>.
chitralverma commented on issue #6235:
URL: https://github.com/apache/opendal/issues/6235#issuecomment-2922666353
after setting up azurite/ minio and testing this locally, i see the issue now,
for local fs, the exception is being thrown at the time of file `open` properly as a `opendal.exceptions.ConditionNotMatch: ConditionNotMatch (permanent)`
for azblob/ s3(minio) your [observation](https://github.com/apache/opendal/issues/6235#issuecomment-2921603821) was correct, and the error comes at the time of file `close`. Debugging the error, i see that its of kind `Interrupted` which wraps the `ConditionNotMatch` error.
<details>
<summary>Error Block</summary>
```rs
&err = Custom {
kind: Interrupted,
error: Error {
kind: ConditionNotMatch,
message: "AzblobError { code: \"BlobAlreadyExists\", message: \"The specified blob already exists. RequestId:5c62cc3a-1cac-422f-87b0-0d5907b337b2 Time:2025-05-30T14:43:22.550Z\" }",
status: Permanent,
operation: "write",
context: [
(
"uri",
"http://127.0.0.1:10000/devstoreaccount1/test/tmp/test_file_37b695e5-3c8e-4110-9f86-7e27ee134441.txt",
),
(
"response",
"Parts { status: 409, version: HTTP/1.1, headers: {\"server\": \"Azurite-Blob/3.23.0\", \"x-ms-error-code\": \"BlobAlreadyExists\", \"x-ms-request-id\": \"5c62cc3a-1cac-422f-87b0-0d5907b337b2\", \"content-type\": \"application/xml\", \"date\": \"Fri, 30 May 2025 14:43:22 GMT\", \"connection\": \"keep-alive\", \"keep-alive\": \"timeout=5\", \"transfer-encoding\": \"chunked\"} }",
),
(
"service",
"azblob",
),
(
"path",
"test_file_37b695e5-3c8e-4110-9f86-7e27ee134441.txt",
),
(
"written",
"0",
),
],
source: None,
},
}
```
</details>
This is happening for async and sync operator because they both return `std::io::{Result, Error}`, `not opendal::Error`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: (services/azblob) `if_not_exist` capability is not working on write [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #6235:
URL: https://github.com/apache/opendal/issues/6235#issuecomment-2922195200
What's the error thrown by fs?
I'm sure this test is covered:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: (services/azblob) `if_not_exist` capability is not working on write [opendal]
Posted by "chitralverma (via GitHub)" <gi...@apache.org>.
chitralverma commented on issue #6235:
URL: https://github.com/apache/opendal/issues/6235#issuecomment-2922405615
what I'm trying to say is that the [pytest is same](https://github.com/apache/opendal/blob/8860387893b2260ac03d0bdc02dda4a983466c43/bindings/python/tests/test_write.py#L219), but for [local_fs the tests do not break](https://github.com/apache/opendal/actions/runs/15340065425/job/43164418365?pr=6086) and for [azblob](https://github.com/apache/opendal/actions/runs/15340065425/job/43164417657?pr=6086) and [s3 (minio)](https://github.com/apache/opendal/actions/runs/15340065425/job/43164422432?pr=6086) same test fails.
so is there a suggestion from your side for this, or is this okay ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: (services/azblob) `if_not_exist` capability is not working on write [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #6235:
URL: https://github.com/apache/opendal/issues/6235#issuecomment-2921734247
> yes seems like it, maybe it's an inconsistency issue? because other services throw condition not match error
I don't think it's an inconsistency issue. Other services are either ignored due to a lack of credentials or because they don't support this feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug(bindings/python): Python bindings can throw different error types for the same error code [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #6235:
URL: https://github.com/apache/opendal/issues/6235#issuecomment-2933226958
I updated the issue title to better reflect the root cause.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org