You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/12/16 17:35:08 UTC
[GitHub] [arrow] pitrou opened a new pull request #8941: ARROW-10942: [C++] Fix S3FileSystem::Impl::IsEmptyDirectory on Amazon
pitrou opened a new pull request #8941:
URL: https://github.com/apache/arrow/pull/8941
Amazon and Minio S3 implementations unfortunately have slightly different conventions for recognizing and handling empty "directories".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] juanjgalvez commented on pull request #8941: ARROW-10942: [C++] Fix S3FileSystem::Impl::IsEmptyDirectory on Amazon
Posted by GitBox <gi...@apache.org>.
juanjgalvez commented on pull request #8941:
URL: https://github.com/apache/arrow/pull/8941#issuecomment-746786024
Btw, tested it and it works.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #8941: ARROW-10942: [C++] Fix S3FileSystem::Impl::IsEmptyDirectory on Amazon
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #8941:
URL: https://github.com/apache/arrow/pull/8941#discussion_r545162804
##########
File path: cpp/src/arrow/filesystem/s3fs.cc
##########
@@ -1241,6 +1245,13 @@ class S3FileSystem::Impl {
return std::string(FromAwsString(builder_.config().region));
}
+ template <typename Error>
+ void SaveBackend(const Aws::Client::AWSError<Error>& error) {
+ if (!backend_ || *backend_ == S3Backend::Other) {
Review comment:
Not necessarily, but if the backend was already detected, we probably don't want to undo the detection if the HTTP header sometimes misses or is incorrect.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #8941: ARROW-10942: [C++] Fix S3FileSystem::Impl::IsEmptyDirectory on Amazon
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #8941:
URL: https://github.com/apache/arrow/pull/8941#issuecomment-747498039
> Would it make sense to do something similar in IsNonEmptyDirectory?
I don't think so. I doesn't use the same S3 API call so so will be subject to different conditions. But if you find a bug with it, feel free to report it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou closed pull request #8941: ARROW-10942: [C++] Fix S3FileSystem::Impl::IsEmptyDirectory on Amazon
Posted by GitBox <gi...@apache.org>.
pitrou closed pull request #8941:
URL: https://github.com/apache/arrow/pull/8941
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on a change in pull request #8941: ARROW-10942: [C++] Fix S3FileSystem::Impl::IsEmptyDirectory on Amazon
Posted by GitBox <gi...@apache.org>.
kou commented on a change in pull request #8941:
URL: https://github.com/apache/arrow/pull/8941#discussion_r544639297
##########
File path: cpp/src/arrow/filesystem/s3fs.cc
##########
@@ -1241,6 +1245,13 @@ class S3FileSystem::Impl {
return std::string(FromAwsString(builder_.config().region));
}
+ template <typename Error>
+ void SaveBackend(const Aws::Client::AWSError<Error>& error) {
+ if (!backend_ || *backend_ == S3Backend::Other) {
Review comment:
Do we need this condition?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8941: ARROW-10942: [C++] Fix S3FileSystem::Impl::IsEmptyDirectory on Amazon
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8941:
URL: https://github.com/apache/arrow/pull/8941#issuecomment-746721298
https://issues.apache.org/jira/browse/ARROW-10942
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org