You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "kou (via GitHub)" <gi...@apache.org> on 2024/04/09 08:20:33 UTC
[I] [C++][FS][Azure] CopyFile doesn't work with Azure hierachical namespace support [arrow]
kou opened a new issue, #41095:
URL: https://github.com/apache/arrow/issues/41095
### Describe the enhancement requested
```text
[ RUN ] TestAzureHierarchicalNSGeneric.CopyFile
/home/kou/work/cpp/arrow.kou/cpp/src/arrow/filesystem/test_util.cc:531: Failure
Failed
'fs->CopyFile("AB/abc", "def")' failed with IOError: Failed to copy a blob. (https://XXX.blob.core.windows.net/a2giu5a9ij1szn7tsx4l0z6194h5zqww/AB/abc -> https://clearcodearrow.blob.core.windows.net/a2giu5a9ij1szn7tsx4l0z6194h5zqww/def) Azure Error: [CannotVerifyCopySource] 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
RequestId:2c08ee06-f01e-0062-12d2-807ca0000000
Time:2024-03-28T05:40:28.1650700Z
Request ID: 2c08ee06-f01e-0062-12d2-807ca0000000
[ FAILED ] TestAzureHierarchicalNSGeneric.CopyFile (1007 ms)
[ OK ] TestAzureHierarchicalNSGeneric.OpenInputFileAsync (394 ms)
[ RUN ] TestAzureHierarchicalNSGeneric.SpecialChars
/home/kou/work/cpp/arrow.kou/cpp/src/arrow/filesystem/test_util.cc:1178: Failure
Failed
'fs->CopyFile("Blank Char/Special%Char.txt", "Special and%different.txt")' failed with IOError: Failed to copy a blob. (https://XXX.blob.core.windows.net/armiaz82i30wliew7324v0l421a4uhop/Blank%20Char/Special%25Char.txt -> https://XXX.blob.core.windows.net/armiaz82i30wliew7324v0l421a4uhop/Special%20and%25different.txt) Azure Error: [CannotVerifyCopySource] 401 Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
RequestId:dbfb99e1-801e-008c-4ad2-80d689000000
Time:2024-03-28T05:40:38.3406267Z
Request ID: dbfb99e1-801e-008c-4ad2-80d689000000
[ FAILED ] TestAzureHierarchicalNSGeneric.SpecialChars (475 ms)
### Component(s)
C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][FS][Azure] CopyFile doesn't work with Azure hierachical namespace support [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on issue #41095:
URL: https://github.com/apache/arrow/issues/41095#issuecomment-2047874882
We use `CopyFromUri` [1]. The issue here doesn't seem to be HNS, but authorizing the source with the same credentials used to create the `arrow::fs::ArrowFileSystem`.
> Server failed to authenticate the request. Please refer to the information in the www-authenticate header.
[1]:
```cpp
Azure::Response<Models::CopyBlobFromUriResult> BlobClient::CopyFromUri(
const std::string& sourceUri,
const CopyBlobFromUriOptions& options,
const Azure::Core::Context& context) const
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][FS][Azure] CopyFile doesn't work with Azure hierachical namespace support [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41095:
URL: https://github.com/apache/arrow/issues/41095#issuecomment-2048454774
> What is `XXX`?
I wanted to mask the account but I missed one substitution...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][FS][Azure] CopyFile doesn't work with Azure hierachical namespace support [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on issue #41095:
URL: https://github.com/apache/arrow/issues/41095#issuecomment-2048585083
Can you report the issue to Microsoft? The function is provided by the SDK and it doesn't seem right that it's failing to authorize the access.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][FS][Azure] CopyFile doesn't work with Azure hierarchical namespace support [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41095:
URL: https://github.com/apache/arrow/issues/41095#issuecomment-2063199043
https://github.com/Azure/azure-sdk-for-cpp/issues/5542#issuecomment-2062908150
> it seems the source URL doesn't contain any authentication information.
>
> You can use SAS token to authenticate the copy source.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][FS][Azure] CopyFile doesn't work with Azure hierachical namespace support [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41095:
URL: https://github.com/apache/arrow/issues/41095#issuecomment-2046654108
https://learn.microsoft.com/en-us/rest/api/storageservices/data-lake-storage-gen2
It seems that `Copy` API doesn't exist...?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][FS][Azure] CopyFile doesn't work with Azure hierarchical namespace support [arrow]
Posted by "Tom-Newton (via GitHub)" <gi...@apache.org>.
Tom-Newton commented on issue #41095:
URL: https://github.com/apache/arrow/issues/41095#issuecomment-2065358742
It's a bit disappointing that Azure doesn't support this without SAS toek, though it does match up with my memory https://github.com/apache/arrow/pull/39058#issuecomment-1838159705
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][FS][Azure] CopyFile doesn't work with Azure hierarchical namespace support [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou closed issue #41095: [C++][FS][Azure] CopyFile doesn't work with Azure hierarchical namespace support
URL: https://github.com/apache/arrow/issues/41095
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][FS][Azure] CopyFile doesn't work with Azure hierarchical namespace support [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41095:
URL: https://github.com/apache/arrow/issues/41095#issuecomment-2068301387
Issue resolved by pull request 41276
https://github.com/apache/arrow/pull/41276
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [C++][FS][Azure] CopyFile doesn't work with Azure hierachical namespace support [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #41095:
URL: https://github.com/apache/arrow/issues/41095#issuecomment-2060480796
I couldn't find whether should we report this... So I've opened a question issue in Azure/azure-sdk-for-cpp: https://github.com/Azure/azure-sdk-for-cpp/issues/5542
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org