You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "TomasRachunek (via GitHub)" <gi...@apache.org> on 2026/04/27 11:19:08 UTC
[I] Missing Artifact ID in bulk delete URL for Fabric Lakehouse over abfss:// [arrow-rs-object-store]
TomasRachunek opened a new issue, #701:
URL: https://github.com/apache/arrow-rs-object-store/issues/701
**Describe the bug**
When a bulk delete is attempted for a Fabric Lakehouse over abfss://, an incorrect Blob URL containing only the Workspace ID and not the Artifact ID is used to issue the bulk deletes, triggering a 400 Bad Request error:
`{"error":{"code":"BadRequest","message":"Either WorkspaceId or ArtifactId are missing in the request"}}`
The incorrect URL format: `https://onelake.blob.fabric.microsoft.com/<WORKSPACE_ID>?restype=container&comp=batch`
Expected URL format: `https://onelake.blob.fabric.microsoft.com/<WORKSPACE_ID>/<ARTIFACT_ID>?restype=container&comp=batch`
**To Reproduce**
Trigger a bulk delete when writing to an Azure Lakehouse through an abfss:// URI.
**Expected behavior**
Bulk delete for Azure Lakehouse functions as intended.
**Additional context**
I have a custom Python data processing pipeline in MS Fabric using `deltalake` which overwrites a table in a Lakehouse using an abfss:// URI.
However, recently, some tables have started experiencing an error with malformed bulk delete URLs being used during a write.
I have managed to trace that this error is caused by URL generation within arrow-rs-object-store.
In the abfss:// case the Artifact/Lakehouse ID is in the first part of the path, but `bulk_delete_request` starts building the URL from the root, stripping the Artifact ID: https://github.com/apache/arrow-rs-object-store/blob/main/src/azure/client.rs#L678
The resulting URL then only contains the Workspace ID and is missing the Artifact ID for the Lakehouse.
Error log:
```
File "/home/trusted-service-user/jupyter-env/python3.11/lib/python3.11/site-packages/deltalake/writer/writer.py", line 131, in write_deltalake
table._table.write(
OSError: Generic MicrosoftAzure error
↳ Error performing bulk delete request
↳ Error performing POST https://onelake.blob.fabric.microsoft.com/<WORKSPACE_ID>?restype=container&comp=batch in 7.561661ms - Server returned non-2xx status code
↳ 400 Bad Request
↳ {"error":{"code":"BadRequest","message":"Either WorkspaceId or ArtifactId are missing in the request"}}
```
Disclosure: I have used AI to trace this issue from the Python code to arrow-rs-object-store, though I have done my best to independently verify the bug in the library and write this report.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Missing Artifact ID in bulk delete URL for Fabric Lakehouse over abfss:// [arrow-rs-object-store]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #701:
URL: https://github.com/apache/arrow-rs-object-store/issues/701#issuecomment-4735295434
@TomasRachunek thanks for pinging again and sorry for the late reply, this was buried in my email.
I work on OneLake and verified with internal folks about Blob bulk delete. It is currently not available and we're active working to enable it. Will provide more update when I hear back and follow up with a fix in this repo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Missing Artifact ID in bulk delete URL for Fabric Lakehouse over abfss:// [arrow-rs-object-store]
Posted by "TomasRachunek (via GitHub)" <gi...@apache.org>.
TomasRachunek commented on issue #701:
URL: https://github.com/apache/arrow-rs-object-store/issues/701#issuecomment-4611744264
@kevinjqliu Hello, sorry for the ping, but is there anything I can do to help further here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Missing Artifact ID in bulk delete URL for Fabric Lakehouse over abfss:// [arrow-rs-object-store]
Posted by "TomasRachunek (via GitHub)" <gi...@apache.org>.
TomasRachunek commented on issue #701:
URL: https://github.com/apache/arrow-rs-object-store/issues/701#issuecomment-4371023925
> Thanks for reporting this! The code you linked does seem suspicious. Do you have the `abfss://` URI used that broke? I want to make sure I'm using the exact same format when I'm reproducing this
Hello, sorry for the late reply.
I'm hesitant to provide real workspace/warehouse UUIDs since I don't know if there's any information leakage possible, so here it is with substituted UUIDs in the same format, which I hope is sufficient:
`abfss://11111111-1111-1111-1111-111111111111@onelake.dfs.fabric.microsoft.com/22222222-2222-2222-2222-222222222222/Tables/SCHEMA/TABLE/`
(where the 1s UUID is the workspace, and the 2s is the lakehouse)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Missing Artifact ID in bulk delete URL for Fabric Lakehouse over abfss:// [arrow-rs-object-store]
Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #701:
URL: https://github.com/apache/arrow-rs-object-store/issues/701#issuecomment-4337329003
Thanks for reporting this! The code you linked does seem suspicious. Do you have the `abfss://` URI used that broke? I want to make sure I'm using the exact same format when I'm reproducing this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Missing Artifact ID in bulk delete URL for Fabric Lakehouse over abfss:// [arrow-rs-object-store]
Posted by "MartyP233 (via GitHub)" <gi...@apache.org>.
MartyP233 commented on issue #701:
URL: https://github.com/apache/arrow-rs-object-store/issues/701#issuecomment-5054297617
Hello, I too am experiencing this issue trying use deltalake > 1.5.0 . Let me know if i can provide any more diagnostic information. I'll write a test that reproduces the error to help me test whether i can upgrade.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org