You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flume.apache.org by Shiva Ram <sh...@gmail.com> on 2015/12/29 08:56:47 UTC

Transfer files from windows server to HDFS

Hi All,

Is there any reliable solution to transfer files from windows server to
HDFS using Flume periodically?

Thanks.
Shivaram.

Re: Transfer files from windows server to HDFS

Posted by Shiva Ram <sh...@gmail.com>.

Thanks Roshan.

Did anyone used Flume with HDFS Sink with kerberos authentication?

Which kerberos principal I have to use?
On 30-Dec-2015 2:15 am, "Roshan Naik" <ro...@hortonworks.com> wrote:

> HDP is the one I am familiar with on Windows.
>
> You could actually try running Apache Flume directly too. Did a quick scan
> of the windows related open jiras.. For the components you need to use, I
> think you might be fine.
>
> -roshan
>
>
> On 12/29/15, 1:58 AM, "Shiva Ram" <sh...@gmail.com> wrote:
>
> >Thanks for your information Roshan.
> >
> >Do we have windows flume distro?
> >
> >Other suggestions required, Is there any reliable solution to transfer
> >files from windows server to HDFS periodically?
> >
> >Thanks.
> >On 29-Dec-2015 3:15 pm, "Roshan Naik" <ro...@hortonworks.com> wrote:
> >
> >> Use any Windows certified Flume distro ... use the SpoolDir source, file
> >> channel and HDFS sink.
> >>
> >> On 12/28/15, 11:56 PM, "Shiva Ram" <sh...@gmail.com>
> >>wrote:
> >>
> >> >Hi All,
> >> >
> >> >Is there any reliable solution to transfer files from windows server to
> >> >HDFS using Flume periodically?
> >> >
> >> >Thanks.
> >> >Shivaram.
> >>
> >>
>
>

Re: Transfer files from windows server to HDFS

Posted by Roshan Naik <ro...@hortonworks.com>.

HDP is the one I am familiar with on Windows.

You could actually try running Apache Flume directly too. Did a quick scan
of the windows related open jiras.. For the components you need to use, I
think you might be fine.

-roshan

On 12/29/15, 1:58 AM, "Shiva Ram" <sh...@gmail.com> wrote:

>Thanks for your information Roshan.
>
>Do we have windows flume distro?
>
>Other suggestions required, Is there any reliable solution to transfer
>files from windows server to HDFS periodically?
>
>Thanks.
>On 29-Dec-2015 3:15 pm, "Roshan Naik" <ro...@hortonworks.com> wrote:
>
>> Use any Windows certified Flume distro ... use the SpoolDir source, file
>> channel and HDFS sink.
>>
>> On 12/28/15, 11:56 PM, "Shiva Ram" <sh...@gmail.com>
>>wrote:
>>
>> >Hi All,
>> >
>> >Is there any reliable solution to transfer files from windows server to
>> >HDFS using Flume periodically?
>> >
>> >Thanks.
>> >Shivaram.
>>
>>

Re: Transfer files from windows server to HDFS

Posted by Shiva Ram <sh...@gmail.com>.

Thanks for your information Roshan.

Do we have windows flume distro?

Other suggestions required, Is there any reliable solution to transfer
files from windows server to HDFS periodically?

Thanks.
On 29-Dec-2015 3:15 pm, "Roshan Naik" <ro...@hortonworks.com> wrote:

> Use any Windows certified Flume distro ... use the SpoolDir source, file
> channel and HDFS sink.
>
> On 12/28/15, 11:56 PM, "Shiva Ram" <sh...@gmail.com> wrote:
>
> >Hi All,
> >
> >Is there any reliable solution to transfer files from windows server to
> >HDFS using Flume periodically?
> >
> >Thanks.
> >Shivaram.
>
>

Re: Transfer files from windows server to HDFS

Posted by Roshan Naik <ro...@hortonworks.com>.

Use any Windows certified Flume distro ... use the SpoolDir source, file
channel and HDFS sink.

On 12/28/15, 11:56 PM, "Shiva Ram" <sh...@gmail.com> wrote:

>Hi All,
>
>Is there any reliable solution to transfer files from windows server to
>HDFS using Flume periodically?
>
>Thanks.
>Shivaram.