You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Wes McKinney (JIRA)" <ji...@apache.org> on 2018/12/18 03:17:00 UTC
[jira] [Commented] (ARROW-4058) [C++] arrow-io-hdfs-test fails when
run against HDFS cluster from docker-compose
[ https://issues.apache.org/jira/browse/ARROW-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16723621#comment-16723621 ]
Wes McKinney commented on ARROW-4058:
-------------------------------------
After some Google searching I found that adding the {{$ARROW_ROOT/integration/hdfs}} directory to {{$CLASSPATH}} made the error go away
{code}
$ export CLASSPATH=/home/wesm/code/arrow/integration/hdfs:$CLASSPATH
(arrow-dev) 21:14 ~/code/arrow/cpp/build (ARROW-2919)$ ./debug/arrow-io-hdfs-testRunning main() from gtest_main.cc
[==========] Running 24 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 12 tests from TestHadoopFileSystem/0, where TypeParam = arrow::io::JNIDriver
[ RUN ] TestHadoopFileSystem/0.ConnectsAgain
[ OK ] TestHadoopFileSystem/0.ConnectsAgain (892 ms)
[ RUN ] TestHadoopFileSystem/0.MultipleClients
[ OK ] TestHadoopFileSystem/0.MultipleClients (178 ms)
[ RUN ] TestHadoopFileSystem/0.MakeDirectory
[ OK ] TestHadoopFileSystem/0.MakeDirectory (134 ms)
[ RUN ] TestHadoopFileSystem/0.GetCapacityUsed
[ OK ] TestHadoopFileSystem/0.GetCapacityUsed (119 ms)
[ RUN ] TestHadoopFileSystem/0.GetPathInfo
[ OK ] TestHadoopFileSystem/0.GetPathInfo (243 ms)
[ RUN ] TestHadoopFileSystem/0.AppendToFile
[ OK ] TestHadoopFileSystem/0.AppendToFile (197 ms)
[ RUN ] TestHadoopFileSystem/0.ListDirectory
[ OK ] TestHadoopFileSystem/0.ListDirectory (167 ms)
[ RUN ] TestHadoopFileSystem/0.ReadableMethods
[ OK ] TestHadoopFileSystem/0.ReadableMethods (164 ms)
[ RUN ] TestHadoopFileSystem/0.LargeFile
[ OK ] TestHadoopFileSystem/0.LargeFile (200 ms)
[ RUN ] TestHadoopFileSystem/0.RenameFile
[ OK ] TestHadoopFileSystem/0.RenameFile (150 ms)
[ OK ] TestHadoopFileSystem/0.ChmodChown (166 ms)
[ RUN ] TestHadoopFileSystem/0.ThreadSafety
[ OK ] TestHadoopFileSystem/0.ThreadSafety (933 ms)
[----------] 12 tests from TestHadoopFileSystem/0 (3544 ms total)
[----------] 12 tests from TestHadoopFileSystem/1, where TypeParam = arrow::io::PivotalDriver
[ RUN ] TestHadoopFileSystem/1.ConnectsAgain
Loading libhdfs3 failed, skipping tests gracefully. IOError: ../src/arrow/io/hdfs-internal.cc:566 code: try_dlopen(libhdfs3_potential_paths, "libhdfs3", shim->handle)
Unable to load libhdfs3
Driver not loaded, skipping
[ OK ] TestHadoopFileSystem/1.ConnectsAgain (0 ms)
[ RUN ] TestHadoopFileSystem/1.MultipleClients
Loading libhdfs3 failed, skipping tests gracefully. IOError: Prior attempt to load libhdfs3 failed
Driver not loaded, skipping
[ OK ] TestHadoopFileSystem/1.MultipleClients (0 ms)
[ RUN ] TestHadoopFileSystem/1.MakeDirectory
Loading libhdfs3 failed, skipping tests gracefully. IOError: Prior attempt to load libhdfs3 failed
Driver not loaded, skipping
[ OK ] TestHadoopFileSystem/1.MakeDirectory (0 ms)
[ RUN ] TestHadoopFileSystem/1.GetCapacityUsed
Loading libhdfs3 failed, skipping tests gracefully. IOError: Prior attempt to load libhdfs3 failed
Driver not loaded, skipping
[ OK ] TestHadoopFileSystem/1.GetCapacityUsed (0 ms)
[ RUN ] TestHadoopFileSystem/1.GetPathInfo
Loading libhdfs3 failed, skipping tests gracefully. IOError: Prior attempt to load libhdfs3 failed
Driver not loaded, skipping
[ OK ] TestHadoopFileSystem/1.GetPathInfo (0 ms)
[ RUN ] TestHadoopFileSystem/1.AppendToFile
Loading libhdfs3 failed, skipping tests gracefully. IOError: Prior attempt to load libhdfs3 failed
Driver not loaded, skipping
[ OK ] TestHadoopFileSystem/1.AppendToFile (0 ms)
[ RUN ] TestHadoopFileSystem/1.ListDirectory
Loading libhdfs3 failed, skipping tests gracefully. IOError: Prior attempt to load libhdfs3 failed
Driver not loaded, skipping
[ OK ] TestHadoopFileSystem/1.ListDirectory (0 ms)
[ RUN ] TestHadoopFileSystem/1.ReadableMethods
Loading libhdfs3 failed, skipping tests gracefully. IOError: Prior attempt to load libhdfs3 failed
Driver not loaded, skipping
[ OK ] TestHadoopFileSystem/1.ReadableMethods (0 ms)
[ RUN ] TestHadoopFileSystem/1.LargeFile
Loading libhdfs3 failed, skipping tests gracefully. IOError: Prior attempt to load libhdfs3 failed
Driver not loaded, skipping
[ OK ] TestHadoopFileSystem/1.LargeFile (0 ms)
[ RUN ] TestHadoopFileSystem/1.RenameFile
Loading libhdfs3 failed, skipping tests gracefully. IOError: Prior attempt to load libhdfs3 failed
Driver not loaded, skipping
[ OK ] TestHadoopFileSystem/1.RenameFile (0 ms)
[ RUN ] TestHadoopFileSystem/1.ChmodChown
Loading libhdfs3 failed, skipping tests gracefully. IOError: Prior attempt to load libhdfs3 failed
Driver not loaded, skipping
[ OK ] TestHadoopFileSystem/1.ChmodChown (0 ms)
[ RUN ] TestHadoopFileSystem/1.ThreadSafety
Loading libhdfs3 failed, skipping tests gracefully. IOError: Prior attempt to load libhdfs3 failed
Driver not loaded, skipping
[ OK ] TestHadoopFileSystem/1.ThreadSafety (0 ms)
[----------] 12 tests from TestHadoopFileSystem/1 (0 ms total)
[----------] Global test environment tear-down
[==========] 24 tests from 2 test cases ran. (3545 ms total)
[ PASSED ] 24 tests.
{code}
This is a serious rough edge for developers, and needs to be carefully documented so people can work on this code reliably
> [C++] arrow-io-hdfs-test fails when run against HDFS cluster from docker-compose
> --------------------------------------------------------------------------------
>
> Key: ARROW-4058
> URL: https://issues.apache.org/jira/browse/ARROW-4058
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Reporter: Wes McKinney
> Priority: Major
> Fix For: 0.12.0
>
>
> When running HDFS with {{docker-compose up hdfs-data-node-1}} I get the following error:
> {code}
> [==========] Running 2 tests from 2 test cases.
> [----------] Global test environment set-up.
> [----------] 1 test from TestHadoopFileSystem/0, where TypeParam = arrow::io::JNIDriver
> [ RUN ] TestHadoopFileSystem/0.LargeFile
> [ OK ] TestHadoopFileSystem/0.LargeFile (1101 ms)
> [----------] 1 test from TestHadoopFileSystem/0 (1101 ms total)
> [----------] 1 test from TestHadoopFileSystem/1, where TypeParam = arrow::io::PivotalDriver
> [ RUN ] TestHadoopFileSystem/1.LargeFile
> Loading libhdfs3 failed, skipping tests gracefully. IOError: ../src/arrow/io/hdfs-internal.cc:566 code: try_dlopen(libhdfs3_potential_paths, "libhdfs3", shim->handle)
> Unable to load libhdfs3
> Driver not loaded, skipping
> [ OK ] TestHadoopFileSystem/1.LargeFile (0 ms)
> [----------] 1 test from TestHadoopFileSystem/1 (0 ms total)
> [----------] Global test environment tear-down
> [==========] 2 tests from 2 test cases ran. (1101 ms total)
> [ PASSED ] 2 tests.
> (arrow-dev) 21:06 ~/code/arrow/cpp/build (ARROW-2919)$ ./debug/arrow-io-hdfs-test
> Running main() from gtest_main.cc
> [==========] Running 24 tests from 2 test cases.
> [----------] Global test environment set-up.
> [----------] 12 tests from TestHadoopFileSystem/0, where TypeParam = arrow::io::JNIDriver
> [ RUN ] TestHadoopFileSystem/0.ConnectsAgain
> [ OK ] TestHadoopFileSystem/0.ConnectsAgain (885 ms)
> [ RUN ] TestHadoopFileSystem/0.MultipleClients
> [ OK ] TestHadoopFileSystem/0.MultipleClients (194 ms)
> [ RUN ] TestHadoopFileSystem/0.MakeDirectory
> [ OK ] TestHadoopFileSystem/0.MakeDirectory (139 ms)
> [ RUN ] TestHadoopFileSystem/0.GetCapacityUsed
> [ OK ] TestHadoopFileSystem/0.GetCapacityUsed (119 ms)
> [ RUN ] TestHadoopFileSystem/0.GetPathInfo
> [ OK ] TestHadoopFileSystem/0.GetPathInfo (229 ms)
> [ RUN ] TestHadoopFileSystem/0.AppendToFile
> 18/12/17 21:06:38 WARN hdfs.DFSClient: DataStreamer Exception
> java.io.IOException: Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try. (Nodes: current=[172.19.0.3:50010], original=[172.19.0.3:50010]). The current failed datanode replacement policy is DEFAULT, and a client may configure this via 'dfs.client.block.write.replace-datanode-on-failure.policy' in its configuration.
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.findNewDatanode(DFSOutputStream.java:1036)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:1102)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1249)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:593)
> FSDataOutputStream#close error:
> java.io.IOException: Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try. (Nodes: current=[172.19.0.3:50010], original=[172.19.0.3:50010]). The current failed datanode replacement policy is DEFAULT, and a client may configure this via 'dfs.client.block.write.replace-datanode-on-failure.policy' in its configuration.
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.findNewDatanode(DFSOutputStream.java:1036)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:1102)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1249)
> at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:593)
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> F1217 21:06:38.871151 25929 hdfs.cc:275] Check failed: impl_->Close().ok()
> *** Check failure stack trace: ***
> Aborted (core dumped)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)