You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@sqoop.apache.org by Wei Yan <yw...@gmail.com> on 2016/02/25 03:18:54 UTC

Does Sqoop support writing to S3?

Hi,

I'm running a Sqoop job in a in-house hadoop cluster, and would like to
output to a S3 location. I didn't find much talking about this type
request. So, does Sqoop support this action?

thanks,
Wei

Re: Does Sqoop support writing to S3?

Posted by Wei Yan <yw...@gmail.com>.

Thanks, Jarcec.

On Sun, Feb 28, 2016 at 8:22 AM, Jarek Jarcec Cecho <ja...@apache.org>
wrote:

> Sqoop 1:
>
> You should be able to configure fs.defaultFS to point to S3. It’s not
> something that we’re testing, so I would expect some limitations.
>
> Sqoop 2:
>
> Have native S3 support:
>
>
> https://github.com/apache/sqoop/blob/sqoop2/docs/src/site/sphinx/user/examples/S3Import.rst
>
> Jarcec
>
> > On Feb 24, 2016, at 6:18 PM, Wei Yan <yw...@gmail.com> wrote:
> >
> > Hi,
> >
> > I'm running a Sqoop job in a in-house hadoop cluster, and would like to
> output to a S3 location. I didn't find much talking about this type
> request. So, does Sqoop support this action?
> >
> > thanks,
> > Wei
>
>

Re: Does Sqoop support writing to S3?

Posted by Jarek Jarcec Cecho <ja...@apache.org>.

Sqoop 1:

You should be able to configure fs.defaultFS to point to S3. It’s not something that we’re testing, so I would expect some limitations.

Sqoop 2:

Have native S3 support:

https://github.com/apache/sqoop/blob/sqoop2/docs/src/site/sphinx/user/examples/S3Import.rst

Jarcec

> On Feb 24, 2016, at 6:18 PM, Wei Yan <yw...@gmail.com> wrote:
> 
> Hi,
> 
> I'm running a Sqoop job in a in-house hadoop cluster, and would like to output to a S3 location. I didn't find much talking about this type request. So, does Sqoop support this action?
> 
> thanks,
> Wei