You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafodion.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/06/29 21:42:00 UTC
[jira] [Commented] (TRAFODION-3126) Refactored HDFS client
implementation should also support Alluxio file system
[ https://issues.apache.org/jira/browse/TRAFODION-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528288#comment-16528288 ]
ASF GitHub Bot commented on TRAFODION-3126:
-------------------------------------------
GitHub user selvaganesang opened a pull request:
https://github.com/apache/trafodion/pull/1626
[TRAFODION-3126] Refactored HDFS client implementation should also su…
…pport Alluxio file system
Alluxio doesn't support direct ByteBuffer access. Circumvented
this problem by using non-direct ByteBuffer to read hdfs files
when it belongs to Alluxio file system.
No need to change the default setting of USE_LIBHDFS for Alluxio to work.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/selvaganesang/trafodion alluxio_support
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/trafodion/pull/1626.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1626
----
commit 2a6cfd1a24d05e1e243919897a11572e68c14d59
Author: selvaganesang <se...@...>
Date: 2018-06-29T21:26:49Z
[TRAFODION-3126] Refactored HDFS client implementation should also support Alluxio file system
Alluxio doesn't support direct ByteBuffer access. Circumvented
this problem by using non-direct ByteBuffer to read hdfs files
when it belongs to Alluxio file system.
No need to change the default setting of USE_LIBHDFS for Alluxio to work.
----
> Refactored HDFS client implementation should also support Alluxio file system
> -----------------------------------------------------------------------------
>
> Key: TRAFODION-3126
> URL: https://issues.apache.org/jira/browse/TRAFODION-3126
> Project: Apache Trafodion
> Issue Type: Sub-task
> Components: sql-exe
> Reporter: Selvaganesan Govindarajan
> Assignee: Selvaganesan Govindarajan
> Priority: Major
>
> New implementation returns the following error when Alluxio file system is involved.
> *** ERROR[8447] An error occurred during hdfs access. Error Detail: SETUP_HDFS_SCAN java.util.concurrent.ExecutionException: java.lang.UnsupportedOperationException: Byte-buffer read unsupported by input stream
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> org.trafodion.sql.HDFSClient.trafHdfsReadBuffer(HDFSClient.java:278)
> org.trafodion.sql.HdfsScan.trafHdfsRead(HdfsScan.java:201) Caused by
> java.lang.UnsupportedOperationException: Byte-buffer read unsupported by input stream
> org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:150)
> org.trafodion.sql.HDFSClient$HDFSRead.call(HDFSClient.java:172)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)