You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/04/21 09:09:54 UTC
[GitHub] [arrow] pitrou opened a new pull request, #12943: ARROW-16261: [C++] Fix DeleteDirContents on HDFS with missing_dir_ok=True
pitrou opened a new pull request, #12943:
URL: https://github.com/apache/arrow/pull/12943
Also improve errno propagation from HDFS-related errors.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kszucs closed pull request #12943: ARROW-16261: [C++] Fix DeleteDirContents on HDFS with missing_dir_ok=True
Posted by GitBox <gi...@apache.org>.
kszucs closed pull request #12943: ARROW-16261: [C++] Fix DeleteDirContents on HDFS with missing_dir_ok=True
URL: https://github.com/apache/arrow/pull/12943
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #12943: ARROW-16261: [C++] Fix DeleteDirContents on HDFS with missing_dir_ok=True
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #12943:
URL: https://github.com/apache/arrow/pull/12943#issuecomment-1112281139
Benchmark runs are scheduled for baseline = fd5f28ec8b73904cc8291fc9d7ee24d1fb0231b6 and contender = c04c3ff288b711641fb5f17d7151ae436796e2b8. c04c3ff288b711641fb5f17d7151ae436796e2b8 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/be39e69b5f234270a34d6bfe3a863171...4283e883d5ec4d9ca2358efb16980836/)
[Failed] [test-mac-arm](https://conbench.ursa.dev/compare/runs/01c1cd149734480b9ee15b3cc9b65c18...b3e754a27c6f4d86a48080baf256abcb/)
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/b67f3b81dafa4620a781cbdfbe827801...af6238c37b6b46659cdb743d6040ea40/)
[Finished :arrow_down:0.34% :arrow_up:0.08%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/6731a6b89b9d45b4a4b9c8c9259d2af4...9f09b51a44244e0484888b83ee02ff79/)
Buildkite builds:
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/610| `c04c3ff2` ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/598| `c04c3ff2` test-mac-arm>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/597| `c04c3ff2` ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/608| `c04c3ff2` ursa-thinkcentre-m75q>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/609| `fd5f28ec` ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/597| `fd5f28ec` test-mac-arm>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/596| `fd5f28ec` ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/607| `fd5f28ec` ursa-thinkcentre-m75q>
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #12943: ARROW-16261: [C++] Fix DeleteDirContents on HDFS with missing_dir_ok=True
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #12943:
URL: https://github.com/apache/arrow/pull/12943#issuecomment-1104923963
https://issues.apache.org/jira/browse/ARROW-16261
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #12943: ARROW-16261: [C++] Fix DeleteDirContents on HDFS with missing_dir_ok=True
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #12943:
URL: https://github.com/apache/arrow/pull/12943#issuecomment-1104970011
Revision: 07412519f959385110f3d38ad152bb625900b771
Submitted crossbow builds: [ursacomputing/crossbow @ actions-1893](https://github.com/ursacomputing/crossbow/branches/all?query=actions-1893)
|Task|Status|
|----|------|
|test-conda-python-3.7-hdfs-2.9.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1893-github-test-conda-python-3.7-hdfs-2.9.2)|
|test-conda-python-3.7-hdfs-3.2.1|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1893-github-test-conda-python-3.7-hdfs-3.2.1)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] westonpace commented on a diff in pull request #12943: ARROW-16261: [C++] Fix DeleteDirContents on HDFS with missing_dir_ok=True
Posted by GitBox <gi...@apache.org>.
westonpace commented on code in PR #12943:
URL: https://github.com/apache/arrow/pull/12943#discussion_r856343619
##########
cpp/src/arrow/filesystem/hdfs.cc:
##########
@@ -194,28 +195,33 @@ class HadoopFileSystem::Impl {
return Status::OK();
}
- Status DeleteDir(const std::string& path) {
- if (!IsDirectory(path)) {
- return Status::IOError("Cannot delete directory '", path, "': not a directory");
+ Status CheckForDirectory(const std::string& path, const char* action) {
+ // Check existence of path, and that it's a directory
+ io::HdfsPathInfo info;
+ RETURN_NOT_OK(client_->GetPathInfo(path, &info));
+ if (info.kind != io::ObjectType::DIRECTORY) {
+ return Status::IOError("Cannot ", action, " directory '", path,
+ "': not a directory");
}
- RETURN_NOT_OK(client_->DeleteDirectory(path));
return Status::OK();
}
+ Status DeleteDir(const std::string& path) {
+ RETURN_NOT_OK(CheckForDirectory(path, "delete"));
+ return client_->DeleteDirectory(path);
+ }
+
Status DeleteDirContents(const std::string& path, bool missing_dir_ok) {
- if (!IsDirectory(path)) {
- return Status::IOError("Cannot delete contents of directory '", path,
- "': not a directory");
- }
- std::vector<std::string> file_list;
- Status get_children_st = client_->GetChildren(path, &file_list);
- if (!get_children_st.ok()) {
- if (missing_dir_ok &&
- ::arrow::internal::ErrnoFromStatus(get_children_st) == ENOENT) {
+ auto st = CheckForDirectory(path, "delete contents of ");
Review Comment:
```suggestion
auto st = CheckForDirectory(path, "delete contents of");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #12943: ARROW-16261: [C++] Fix DeleteDirContents on HDFS with missing_dir_ok=True
Posted by GitBox <gi...@apache.org>.
pitrou commented on PR #12943:
URL: https://github.com/apache/arrow/pull/12943#issuecomment-1104923541
@github-actions crossbow submit *hdfs*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org