You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by yiyu jia <ji...@gmail.com> on 2012/11/27 01:17:09 UTC
Anybody know how to configure SSH for eclipse plugin
Hi,
Anybody tell me how to configure SSH for eclipse plugin? I guess eclipse
plugin use SSH to connect with Map/Reduce locations. But, I found that it
always use my local machine' s account name to connect with hadoop host
servers.
thanks and regards,
Yiyu
--
******************************************
* Mr. Jia Yiyu *
* *
* Email: jia.yiyu@gmail.com *
* *
* Web: http://yiyujia.blogspot.com/*
*******************************************
Re: Anybody know how to configure SSH for eclipse plugin
Posted by Harsh J <ha...@cloudera.com>.
Interesting, so this used to be present but isn't anymore? In any case, the
new project and the members behind it are your best bet for further
enhancements.
On Wed, Nov 28, 2012 at 2:09 AM, yiyu jia <ji...@gmail.com> wrote:
> hi Harsh,
>
> thank you for helps! I think I was blur yesterday. But, it is partly
> misleaded by the comments in HadoopServer.java file
>
>
> * <p>
> * This class does not create any SSH connection anymore. Tunneling must be
> * setup outside of Eclipse for now (using Putty or <tt>ssh -D<port>
> * <host></tt>)
> *
>
> thanks again!
>
> yiyu
>
>
> On Tue, Nov 27, 2012 at 3:16 PM, Harsh J <ha...@cloudera.com> wrote:
>
> > Hi,
> >
> > The Eclipse plugin communicates directly with the RPC servers of NN, DNs,
> > and JT. It does not, in any way, use SSH to do this.
> >
> > If your machine can't setup a fully functional connection to the cluster,
> > then the Eclipse plugin will currently not work and you have to fall back
> > into doing the manual compile-jar -> scp -> invoke cycle, or rely on a
> > local MR cluster (pseudo-distributed) or the local job runner
> (standalone)
> > to test your programs.
> >
> > There is a proposal for splitting out the Eclipse plugin code to its own
> > project, at
> > http://wiki.apache.org/incubator/HadoopDevelopmentToolsProposal,
> > and adding further enhancements to it. Perhaps when this project is
> setup,
> > you can request for such a feature where the plugin may itself automate
> the
> > jar production plus scp and invoke it at the other end - for such
> > scenarios.
> >
> >
> > On Wed, Nov 28, 2012 at 1:25 AM, yiyu jia <ji...@gmail.com> wrote:
> >
> > > Hi Glen,
> > >
> > > Thanks a lot for response! I tried last night by setup ssh tunnel for
> > > certain ports (like 9000 and 9001). Still not successful. My SSH
> > > communication among servers are set to be passwordless with
> certification
> > > file.
> > >
> > > I suspect eclipse plugin does not support SSH with password or
> > > certification files. Anybody can confirm if my suspection is right or
> > > wrong?
> > >
> > > thanks a lot!
> > >
> > > Yiyu
> > >
> > > On Tue, Nov 27, 2012 at 9:44 AM, Glen Mazza <gm...@talend.com> wrote:
> > >
> > > > Unsure, perhaps better to ask on the user@hadoop.apache.org list.
> > > >
> > > > Glen
> > > >
> > > >
> > > > On 11/26/2012 07:17 PM, yiyu jia wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> Anybody tell me how to configure SSH for eclipse plugin? I guess
> > eclipse
> > > >> plugin use SSH to connect with Map/Reduce locations. But, I found
> that
> > > it
> > > >> always use my local machine' s account name to connect with hadoop
> > host
> > > >> servers.
> > > >>
> > > >> thanks and regards,
> > > >>
> > > >> Yiyu
> > > >>
> > > >>
> > > >>
> > > >>
> > > >
> > > > --
> > > > Glen Mazza
> > > > Talend Community Coders - coders.talend.com
> > > > blog: www.jroller.com/gmazza
> > > >
> > > >
> > >
> > >
> > > --
> > > ******************************************
> > > * Mr. Jia Yiyu *
> > > * *
> > > * Email: jia.yiyu@gmail.com *
> > > * *
> > > * Web: http://yiyujia.blogspot.com/*
> > > *******************************************
> > >
> >
> >
> >
> > --
> > Harsh J
> >
>
>
>
> --
> ******************************************
> * Mr. Jia Yiyu *
> * *
> * Email: jia.yiyu@gmail.com *
> * *
> * Web: http://yiyujia.blogspot.com/*
> *******************************************
>
--
Harsh J
Re: Anybody know how to configure SSH for eclipse plugin
Posted by yiyu jia <ji...@gmail.com>.
Hi Andy,
thank you for info. But we do not need to use SSH if my client has no
problem to communicate with NN and TN right?
Also, I notice that this eclipse plugin always send my windows account name
to access HDFS. it keeps doing same thing even I change the
eclipse.plug-inuser.name in teh DFS edition panel. Do you know if the
problem from plugin itself or eclipse?
thank you!
Yiyu
On Tue, Nov 27, 2012 at 3:51 PM, Andy Isaacson <ad...@cloudera.com> wrote:
> If you can configure Eclipse to use a SOCKS5 proxy, then ssh -D may work.
>
> -andy
>
> On Tue, Nov 27, 2012 at 12:39 PM, yiyu jia <ji...@gmail.com> wrote:
> > hi Harsh,
> >
> > thank you for helps! I think I was blur yesterday. But, it is partly
> > misleaded by the comments in HadoopServer.java file
> >
> >
> > * <p>
> > * This class does not create any SSH connection anymore. Tunneling must
> be
> > * setup outside of Eclipse for now (using Putty or <tt>ssh
> -D<port>
> > * <host></tt>)
> > *
> >
> > thanks again!
> >
> > yiyu
> >
> >
> > On Tue, Nov 27, 2012 at 3:16 PM, Harsh J <ha...@cloudera.com> wrote:
> >
> >> Hi,
> >>
> >> The Eclipse plugin communicates directly with the RPC servers of NN,
> DNs,
> >> and JT. It does not, in any way, use SSH to do this.
> >>
> >> If your machine can't setup a fully functional connection to the
> cluster,
> >> then the Eclipse plugin will currently not work and you have to fall
> back
> >> into doing the manual compile-jar -> scp -> invoke cycle, or rely on a
> >> local MR cluster (pseudo-distributed) or the local job runner
> (standalone)
> >> to test your programs.
> >>
> >> There is a proposal for splitting out the Eclipse plugin code to its own
> >> project, at
> >> http://wiki.apache.org/incubator/HadoopDevelopmentToolsProposal,
> >> and adding further enhancements to it. Perhaps when this project is
> setup,
> >> you can request for such a feature where the plugin may itself automate
> the
> >> jar production plus scp and invoke it at the other end - for such
> >> scenarios.
> >>
> >>
> >> On Wed, Nov 28, 2012 at 1:25 AM, yiyu jia <ji...@gmail.com> wrote:
> >>
> >> > Hi Glen,
> >> >
> >> > Thanks a lot for response! I tried last night by setup ssh tunnel for
> >> > certain ports (like 9000 and 9001). Still not successful. My SSH
> >> > communication among servers are set to be passwordless with
> certification
> >> > file.
> >> >
> >> > I suspect eclipse plugin does not support SSH with password or
> >> > certification files. Anybody can confirm if my suspection is right or
> >> > wrong?
> >> >
> >> > thanks a lot!
> >> >
> >> > Yiyu
> >> >
> >> > On Tue, Nov 27, 2012 at 9:44 AM, Glen Mazza <gm...@talend.com>
> wrote:
> >> >
> >> > > Unsure, perhaps better to ask on the user@hadoop.apache.org list.
> >> > >
> >> > > Glen
> >> > >
> >> > >
> >> > > On 11/26/2012 07:17 PM, yiyu jia wrote:
> >> > >
> >> > >> Hi,
> >> > >>
> >> > >> Anybody tell me how to configure SSH for eclipse plugin? I guess
> >> eclipse
> >> > >> plugin use SSH to connect with Map/Reduce locations. But, I found
> that
> >> > it
> >> > >> always use my local machine' s account name to connect with hadoop
> >> host
> >> > >> servers.
> >> > >>
> >> > >> thanks and regards,
> >> > >>
> >> > >> Yiyu
> >> > >>
> >> > >>
> >> > >>
> >> > >>
> >> > >
> >> > > --
> >> > > Glen Mazza
> >> > > Talend Community Coders - coders.talend.com
> >> > > blog: www.jroller.com/gmazza
> >> > >
> >> > >
> >> >
> >> >
> >> > --
> >> > ******************************************
> >> > * Mr. Jia Yiyu *
> >> > * *
> >> > * Email: jia.yiyu@gmail.com *
> >> > * *
> >> > * Web: http://yiyujia.blogspot.com/*
> >> > *******************************************
> >> >
> >>
> >>
> >>
> >> --
> >> Harsh J
> >>
> >
> >
> >
> > --
> > ******************************************
> > * Mr. Jia Yiyu *
> > * *
> > * Email: jia.yiyu@gmail.com *
> > * *
> > * Web: http://yiyujia.blogspot.com/*
> > *******************************************
>
Re: Anybody know how to configure SSH for eclipse plugin
Posted by Andy Isaacson <ad...@cloudera.com>.
If you can configure Eclipse to use a SOCKS5 proxy, then ssh -D may work.
-andy
On Tue, Nov 27, 2012 at 12:39 PM, yiyu jia <ji...@gmail.com> wrote:
> hi Harsh,
>
> thank you for helps! I think I was blur yesterday. But, it is partly
> misleaded by the comments in HadoopServer.java file
>
>
> * <p>
> * This class does not create any SSH connection anymore. Tunneling must be
> * setup outside of Eclipse for now (using Putty or <tt>ssh -D<port>
> * <host></tt>)
> *
>
> thanks again!
>
> yiyu
>
>
> On Tue, Nov 27, 2012 at 3:16 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi,
>>
>> The Eclipse plugin communicates directly with the RPC servers of NN, DNs,
>> and JT. It does not, in any way, use SSH to do this.
>>
>> If your machine can't setup a fully functional connection to the cluster,
>> then the Eclipse plugin will currently not work and you have to fall back
>> into doing the manual compile-jar -> scp -> invoke cycle, or rely on a
>> local MR cluster (pseudo-distributed) or the local job runner (standalone)
>> to test your programs.
>>
>> There is a proposal for splitting out the Eclipse plugin code to its own
>> project, at
>> http://wiki.apache.org/incubator/HadoopDevelopmentToolsProposal,
>> and adding further enhancements to it. Perhaps when this project is setup,
>> you can request for such a feature where the plugin may itself automate the
>> jar production plus scp and invoke it at the other end - for such
>> scenarios.
>>
>>
>> On Wed, Nov 28, 2012 at 1:25 AM, yiyu jia <ji...@gmail.com> wrote:
>>
>> > Hi Glen,
>> >
>> > Thanks a lot for response! I tried last night by setup ssh tunnel for
>> > certain ports (like 9000 and 9001). Still not successful. My SSH
>> > communication among servers are set to be passwordless with certification
>> > file.
>> >
>> > I suspect eclipse plugin does not support SSH with password or
>> > certification files. Anybody can confirm if my suspection is right or
>> > wrong?
>> >
>> > thanks a lot!
>> >
>> > Yiyu
>> >
>> > On Tue, Nov 27, 2012 at 9:44 AM, Glen Mazza <gm...@talend.com> wrote:
>> >
>> > > Unsure, perhaps better to ask on the user@hadoop.apache.org list.
>> > >
>> > > Glen
>> > >
>> > >
>> > > On 11/26/2012 07:17 PM, yiyu jia wrote:
>> > >
>> > >> Hi,
>> > >>
>> > >> Anybody tell me how to configure SSH for eclipse plugin? I guess
>> eclipse
>> > >> plugin use SSH to connect with Map/Reduce locations. But, I found that
>> > it
>> > >> always use my local machine' s account name to connect with hadoop
>> host
>> > >> servers.
>> > >>
>> > >> thanks and regards,
>> > >>
>> > >> Yiyu
>> > >>
>> > >>
>> > >>
>> > >>
>> > >
>> > > --
>> > > Glen Mazza
>> > > Talend Community Coders - coders.talend.com
>> > > blog: www.jroller.com/gmazza
>> > >
>> > >
>> >
>> >
>> > --
>> > ******************************************
>> > * Mr. Jia Yiyu *
>> > * *
>> > * Email: jia.yiyu@gmail.com *
>> > * *
>> > * Web: http://yiyujia.blogspot.com/*
>> > *******************************************
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> ******************************************
> * Mr. Jia Yiyu *
> * *
> * Email: jia.yiyu@gmail.com *
> * *
> * Web: http://yiyujia.blogspot.com/*
> *******************************************
Re: Anybody know how to configure SSH for eclipse plugin
Posted by yiyu jia <ji...@gmail.com>.
hi Harsh,
thank you for helps! I think I was blur yesterday. But, it is partly
misleaded by the comments in HadoopServer.java file
* <p>
* This class does not create any SSH connection anymore. Tunneling must be
* setup outside of Eclipse for now (using Putty or <tt>ssh -D<port>
* <host></tt>)
*
thanks again!
yiyu
On Tue, Nov 27, 2012 at 3:16 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> The Eclipse plugin communicates directly with the RPC servers of NN, DNs,
> and JT. It does not, in any way, use SSH to do this.
>
> If your machine can't setup a fully functional connection to the cluster,
> then the Eclipse plugin will currently not work and you have to fall back
> into doing the manual compile-jar -> scp -> invoke cycle, or rely on a
> local MR cluster (pseudo-distributed) or the local job runner (standalone)
> to test your programs.
>
> There is a proposal for splitting out the Eclipse plugin code to its own
> project, at
> http://wiki.apache.org/incubator/HadoopDevelopmentToolsProposal,
> and adding further enhancements to it. Perhaps when this project is setup,
> you can request for such a feature where the plugin may itself automate the
> jar production plus scp and invoke it at the other end - for such
> scenarios.
>
>
> On Wed, Nov 28, 2012 at 1:25 AM, yiyu jia <ji...@gmail.com> wrote:
>
> > Hi Glen,
> >
> > Thanks a lot for response! I tried last night by setup ssh tunnel for
> > certain ports (like 9000 and 9001). Still not successful. My SSH
> > communication among servers are set to be passwordless with certification
> > file.
> >
> > I suspect eclipse plugin does not support SSH with password or
> > certification files. Anybody can confirm if my suspection is right or
> > wrong?
> >
> > thanks a lot!
> >
> > Yiyu
> >
> > On Tue, Nov 27, 2012 at 9:44 AM, Glen Mazza <gm...@talend.com> wrote:
> >
> > > Unsure, perhaps better to ask on the user@hadoop.apache.org list.
> > >
> > > Glen
> > >
> > >
> > > On 11/26/2012 07:17 PM, yiyu jia wrote:
> > >
> > >> Hi,
> > >>
> > >> Anybody tell me how to configure SSH for eclipse plugin? I guess
> eclipse
> > >> plugin use SSH to connect with Map/Reduce locations. But, I found that
> > it
> > >> always use my local machine' s account name to connect with hadoop
> host
> > >> servers.
> > >>
> > >> thanks and regards,
> > >>
> > >> Yiyu
> > >>
> > >>
> > >>
> > >>
> > >
> > > --
> > > Glen Mazza
> > > Talend Community Coders - coders.talend.com
> > > blog: www.jroller.com/gmazza
> > >
> > >
> >
> >
> > --
> > ******************************************
> > * Mr. Jia Yiyu *
> > * *
> > * Email: jia.yiyu@gmail.com *
> > * *
> > * Web: http://yiyujia.blogspot.com/*
> > *******************************************
> >
>
>
>
> --
> Harsh J
>
--
******************************************
* Mr. Jia Yiyu *
* *
* Email: jia.yiyu@gmail.com *
* *
* Web: http://yiyujia.blogspot.com/*
*******************************************
Re: Anybody know how to configure SSH for eclipse plugin
Posted by Harsh J <ha...@cloudera.com>.
Hi,
The Eclipse plugin communicates directly with the RPC servers of NN, DNs,
and JT. It does not, in any way, use SSH to do this.
If your machine can't setup a fully functional connection to the cluster,
then the Eclipse plugin will currently not work and you have to fall back
into doing the manual compile-jar -> scp -> invoke cycle, or rely on a
local MR cluster (pseudo-distributed) or the local job runner (standalone)
to test your programs.
There is a proposal for splitting out the Eclipse plugin code to its own
project, at http://wiki.apache.org/incubator/HadoopDevelopmentToolsProposal,
and adding further enhancements to it. Perhaps when this project is setup,
you can request for such a feature where the plugin may itself automate the
jar production plus scp and invoke it at the other end - for such scenarios.
On Wed, Nov 28, 2012 at 1:25 AM, yiyu jia <ji...@gmail.com> wrote:
> Hi Glen,
>
> Thanks a lot for response! I tried last night by setup ssh tunnel for
> certain ports (like 9000 and 9001). Still not successful. My SSH
> communication among servers are set to be passwordless with certification
> file.
>
> I suspect eclipse plugin does not support SSH with password or
> certification files. Anybody can confirm if my suspection is right or
> wrong?
>
> thanks a lot!
>
> Yiyu
>
> On Tue, Nov 27, 2012 at 9:44 AM, Glen Mazza <gm...@talend.com> wrote:
>
> > Unsure, perhaps better to ask on the user@hadoop.apache.org list.
> >
> > Glen
> >
> >
> > On 11/26/2012 07:17 PM, yiyu jia wrote:
> >
> >> Hi,
> >>
> >> Anybody tell me how to configure SSH for eclipse plugin? I guess eclipse
> >> plugin use SSH to connect with Map/Reduce locations. But, I found that
> it
> >> always use my local machine' s account name to connect with hadoop host
> >> servers.
> >>
> >> thanks and regards,
> >>
> >> Yiyu
> >>
> >>
> >>
> >>
> >
> > --
> > Glen Mazza
> > Talend Community Coders - coders.talend.com
> > blog: www.jroller.com/gmazza
> >
> >
>
>
> --
> ******************************************
> * Mr. Jia Yiyu *
> * *
> * Email: jia.yiyu@gmail.com *
> * *
> * Web: http://yiyujia.blogspot.com/*
> *******************************************
>
--
Harsh J
Re: Anybody know how to configure SSH for eclipse plugin
Posted by yiyu jia <ji...@gmail.com>.
Hi Glen,
Thanks a lot for response! I tried last night by setup ssh tunnel for
certain ports (like 9000 and 9001). Still not successful. My SSH
communication among servers are set to be passwordless with certification
file.
I suspect eclipse plugin does not support SSH with password or
certification files. Anybody can confirm if my suspection is right or wrong?
thanks a lot!
Yiyu
On Tue, Nov 27, 2012 at 9:44 AM, Glen Mazza <gm...@talend.com> wrote:
> Unsure, perhaps better to ask on the user@hadoop.apache.org list.
>
> Glen
>
>
> On 11/26/2012 07:17 PM, yiyu jia wrote:
>
>> Hi,
>>
>> Anybody tell me how to configure SSH for eclipse plugin? I guess eclipse
>> plugin use SSH to connect with Map/Reduce locations. But, I found that it
>> always use my local machine' s account name to connect with hadoop host
>> servers.
>>
>> thanks and regards,
>>
>> Yiyu
>>
>>
>>
>>
>
> --
> Glen Mazza
> Talend Community Coders - coders.talend.com
> blog: www.jroller.com/gmazza
>
>
--
******************************************
* Mr. Jia Yiyu *
* *
* Email: jia.yiyu@gmail.com *
* *
* Web: http://yiyujia.blogspot.com/*
*******************************************
Re: Anybody know how to configure SSH for eclipse plugin
Posted by Glen Mazza <gm...@talend.com>.
Unsure, perhaps better to ask on the user@hadoop.apache.org list.
Glen
On 11/26/2012 07:17 PM, yiyu jia wrote:
> Hi,
>
> Anybody tell me how to configure SSH for eclipse plugin? I guess eclipse
> plugin use SSH to connect with Map/Reduce locations. But, I found that it
> always use my local machine' s account name to connect with hadoop host
> servers.
>
> thanks and regards,
>
> Yiyu
>
>
>
--
Glen Mazza
Talend Community Coders - coders.talend.com
blog: www.jroller.com/gmazza
Re: Anybody know how to configure SSH for eclipse plugin
Posted by yiyu jia <ji...@gmail.com>.
Hi all,
how to make eclipse plugin to support SSH connection that need password (or
using certification stored somewhere)?
thanks in advance!
Yiyu
On Mon, Nov 26, 2012 at 9:08 PM, yiyu jia <ji...@gmail.com> wrote:
> In file org.apache.hadoop.eclipse.server.HadoopServer.java (version 1.0.4)
> I find the following comments. Does it mean I can setup a SSH tunnel and
> eclipse can connect to remove cluster main node? Any help will be
> appreciated!
>
> * <p>
> * This class does not create any SSH connection anymore. Tunneling must be
> * setup outside of Eclipse for now (using Putty or <tt>ssh -D<port>
> * <host></tt>)
> *
>
>
>
> On Mon, Nov 26, 2012 at 7:17 PM, yiyu jia <ji...@gmail.com> wrote:
>
>>
>> Hi,
>>
>> Anybody tell me how to configure SSH for eclipse plugin? I guess eclipse
>> plugin use SSH to connect with Map/Reduce locations. But, I found that it
>> always use my local machine' s account name to connect with hadoop host
>> servers.
>>
>> thanks and regards,
>>
>> Yiyu
>>
>>
>>
>> --
>> ******************************************
>> * Mr. Jia Yiyu *
>> * *
>> * Email: jia.yiyu@gmail.com *
>> * *
>> * Web: http://yiyujia.blogspot.com/*
>>
>> *******************************************
>>
>
>
>
--
******************************************
* Mr. Jia Yiyu *
* *
* Email: jia.yiyu@gmail.com *
* *
* Web: http://yiyujia.blogspot.com/*
*******************************************
Re: Anybody know how to configure SSH for eclipse plugin
Posted by yiyu jia <ji...@gmail.com>.
In file org.apache.hadoop.eclipse.server.HadoopServer.java (version 1.0.4)
I find the following comments. Does it mean I can setup a SSH tunnel and
eclipse can connect to remove cluster main node? Any help will be
appreciated!
* <p>
* This class does not create any SSH connection anymore. Tunneling must be
* setup outside of Eclipse for now (using Putty or <tt>ssh -D<port>
* <host></tt>)
*
On Mon, Nov 26, 2012 at 7:17 PM, yiyu jia <ji...@gmail.com> wrote:
>
> Hi,
>
> Anybody tell me how to configure SSH for eclipse plugin? I guess eclipse
> plugin use SSH to connect with Map/Reduce locations. But, I found that it
> always use my local machine' s account name to connect with hadoop host
> servers.
>
> thanks and regards,
>
> Yiyu
>
>
>
> --
> ******************************************
> * Mr. Jia Yiyu *
> * *
> * Email: jia.yiyu@gmail.com *
> * *
> * Web: http://yiyujia.blogspot.com/*
>
> *******************************************
>