You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Sergey Bahchissaraitsev (JIRA)" <ji...@apache.org> on 2017/06/19 16:23:00 UTC

[jira] [Commented] (HADOOP-5732) Add SFTP FileSystem

    [ https://issues.apache.org/jira/browse/HADOOP-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054270#comment-16054270 ] 

Sergey Bahchissaraitsev commented on HADOOP-5732:
-------------------------------------------------

I have both configured on Hadoop 2.8.0:
{code:java}
fs.AbstractFileSystem.sftp.impl=org.apache.hadoop.fs.sftp.SFTPFileSystem
fs.sftp.impl=org.apache.hadoop.fs.sftp.SFTPFileSystem
{code}

I get the following error:
{code:java}
org.apache.hadoop.mapred.YarnChild: Exception running child : java.io.IOException: Seek not supported
	at org.apache.hadoop.fs.sftp.SFTPInputStream.seek(SFTPInputStream.java:66)
	at org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:65)
	at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:114)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initialize(PigRecordReader.java:181)
	at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:548)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:786)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
{code}

By looking at the code, it looks like the seek method is unsupported. Not sure what I am missing:
https://github.com/apache/hadoop/blob/release-2.8.0-RC3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPInputStream.java#L66

> Add SFTP FileSystem
> -------------------
>
>                 Key: HADOOP-5732
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5732
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs
>         Environment: Any environment
>            Reporter: Íñigo Goiri
>            Assignee: ramtin
>            Priority: Minor
>             Fix For: 2.8.0, 3.0.0-alpha1
>
>         Attachments: HADOOP-5732.008.patch, HADOOP-5732.009.patch, HADOOP-5732.010.patch, HADOOP-5732.patch, HADOOP-5732.patch, HADOOP-5732.patch, HADOOP-5732.patch, HADOOP-5732.patch, ivy-for-hadoop-7532.patch, ivy-for-hadoop-7532.patch
>
>   Original Estimate: 0h
>  Remaining Estimate: 0h
>
> I have implemented a FileSystem that supports SFTP. It uses JSch (http://www.jcraft.com/jsch/) in order to manage SFTP.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org