using sqoop over proxies

[Sumit Kumar <k....@gmail.com>]

All,

I've a usecase where i need to import and export data to mysql instances
across two different infrastructures (lets call them DC1 and DC2).
Connectivity between the two datacenters is mostly over ssh or ssh
tunneling. Has anyone made sqoop work with ssh proxies that too in
production? Since this import and export step is part of a big oozie
workflow; i was wondering what is the right way to do it (i'm being told
internally that these tunnels are not reliable for production use). Anybody
has a better suggestion or experience getting sqoop to work for such a
usecase?

Thanks,
-Sumit

Re: using sqoop over proxies

[Jarek Jarcec Cecho <ja...@apache.org>]

I'm afraid that sqoop requires ability to connect directly to your database from all nodes in the cluster. 

One solution that crossed my mind - maybe db proxies might help you? I mean you theoretically could deploy your proxy in DC1, configure it to somehow to connect to DC1 and then from DC1 execute sqoop pointing to this (local) proxy.

Jarcec

On Fri, Oct 19, 2012 at 03:07:43PM +0530, Sumit Kumar wrote:
> All,
> 
> I've a usecase where i need to import and export data to mysql instances
> across two different infrastructures (lets call them DC1 and DC2).
> Connectivity between the two datacenters is mostly over ssh or ssh
> tunneling. Has anyone made sqoop work with ssh proxies that too in
> production? Since this import and export step is part of a big oozie
> workflow; i was wondering what is the right way to do it (i'm being told
> internally that these tunnels are not reliable for production use). Anybody
> has a better suggestion or experience getting sqoop to work for such a
> usecase?
> 
> Thanks,
> -Sumit