You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2021/04/08 13:28:27 UTC
[GitHub] [arrow] pitrou opened a new pull request #9949: ARROW-6248: [Python][C++] Raise better exception on HDFS file open error
pitrou opened a new pull request #9949:
URL: https://github.com/apache/arrow/pull/9949
libhdfs sets `errno` to a value corresponding to the Java exception
(see hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/exception.c in Hadoop repo),
we can use that value to return a more informative error to the user.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #9949: ARROW-6248: [Python][C++] Raise better exception on HDFS file open error
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on a change in pull request #9949:
URL: https://github.com/apache/arrow/pull/9949#discussion_r611838018
##########
File path: cpp/src/arrow/io/hdfs.cc
##########
@@ -497,12 +501,15 @@ class HadoopFileSystem::HadoopFileSystemImpl {
Status OpenReadable(const std::string& path, int32_t buffer_size,
const io::IOContext& io_context,
std::shared_ptr<HdfsReadableFile>* file) {
+ errno = 0;
hdfsFile handle = driver_->OpenFile(fs_, path.c_str(), O_RDONLY, buffer_size, 0, 0);
if (handle == nullptr) {
- const char* msg = !Exists(path) ? "HDFS file does not exist: "
- : "HDFS path exists, but opening file failed: ";
- return Status::IOError(msg, path);
+ if (errno) {
Review comment:
Aha, I see. I assume that's standard enough C knowledge to not require a comment ;)
No further comments on the PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #9949: ARROW-6248: [Python][C++] Raise better exception on HDFS file open error
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #9949:
URL: https://github.com/apache/arrow/pull/9949#issuecomment-815996191
Revision: 8ad2e4fa1e0eced2b5836347a4dce6db076b0dc7
Submitted crossbow builds: [ursacomputing/crossbow @ actions-296](https://github.com/ursacomputing/crossbow/branches/all?query=actions-296)
|Task|Status|
|----|------|
|test-conda-python-3.7-hdfs-3.2|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-296-github-test-conda-python-3.7-hdfs-3.2)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #9949: ARROW-6248: [Python][C++] Raise better exception on HDFS file open error
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #9949:
URL: https://github.com/apache/arrow/pull/9949#issuecomment-815988076
https://issues.apache.org/jira/browse/ARROW-6248
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm closed pull request #9949: ARROW-6248: [Python][C++] Raise better exception on HDFS file open error
Posted by GitBox <gi...@apache.org>.
lidavidm closed pull request #9949:
URL: https://github.com/apache/arrow/pull/9949
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a change in pull request #9949: ARROW-6248: [Python][C++] Raise better exception on HDFS file open error
Posted by GitBox <gi...@apache.org>.
pitrou commented on a change in pull request #9949:
URL: https://github.com/apache/arrow/pull/9949#discussion_r611835654
##########
File path: cpp/src/arrow/io/hdfs.cc
##########
@@ -497,12 +501,15 @@ class HadoopFileSystem::HadoopFileSystemImpl {
Status OpenReadable(const std::string& path, int32_t buffer_size,
const io::IOContext& io_context,
std::shared_ptr<HdfsReadableFile>* file) {
+ errno = 0;
hdfsFile handle = driver_->OpenFile(fs_, path.c_str(), O_RDONLY, buffer_size, 0, 0);
if (handle == nullptr) {
- const char* msg = !Exists(path) ? "HDFS file does not exist: "
- : "HDFS path exists, but opening file failed: ";
- return Status::IOError(msg, path);
+ if (errno) {
Review comment:
This is the pseudo-magic C `errno` variable :-)
See https://pubs.opengroup.org/onlinepubs/9699919799/functions/errno.html, https://en.cppreference.com/w/c/error/errno
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorisvandenbossche commented on a change in pull request #9949: ARROW-6248: [Python][C++] Raise better exception on HDFS file open error
Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on a change in pull request #9949:
URL: https://github.com/apache/arrow/pull/9949#discussion_r611828323
##########
File path: cpp/src/arrow/io/hdfs.cc
##########
@@ -497,12 +501,15 @@ class HadoopFileSystem::HadoopFileSystemImpl {
Status OpenReadable(const std::string& path, int32_t buffer_size,
const io::IOContext& io_context,
std::shared_ptr<HdfsReadableFile>* file) {
+ errno = 0;
hdfsFile handle = driver_->OpenFile(fs_, path.c_str(), O_RDONLY, buffer_size, 0, 0);
if (handle == nullptr) {
- const char* msg = !Exists(path) ? "HDFS file does not exist: "
- : "HDFS path exists, but opening file failed: ";
- return Status::IOError(msg, path);
+ if (errno) {
Review comment:
Dummy question, since the `errno` variable is never passed to a libhdfs method, how can this ever have another value than 0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #9949: ARROW-6248: [Python][C++] Raise better exception on HDFS file open error
Posted by GitBox <gi...@apache.org>.
pitrou commented on pull request #9949:
URL: https://github.com/apache/arrow/pull/9949#issuecomment-815829489
@github-actions crossbow submit test-conda-python-3.7-hdfs-3.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org