You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Karim Awara <ka...@kaust.edu.sa> on 2014/05/11 15:57:21 UTC
writing multiple files on hdfs
Hi,
Can I open multiple files on hdfs and write data to them in parallel and
then close them at the end?
--
Best Regards,
Karim Ahmed Awara
--
------------------------------
This message and its contents, including attachments are intended solely
for the original recipient. If you are not the intended recipient or have
received this message in error, please notify me immediately and delete
this message from your computer system. Any unauthorized use or
distribution is prohibited. Please consider the environment before printing
this email.
Re: writing multiple files on hdfs
Posted by Krishna Chaitanya <bn...@gmail.com>.
Yes!
On May 14, 2014 10:34 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Raj K Singh <ra...@gmail.com>.
have u tried the hdfs api to create or write a file to the HDFS, will it
preventing u opening multiple OutputStream on the FileSystem?
::::::::::::::::::::::::::::::::::::::::
Raj K Singh
http://in.linkedin.com/in/rajkrrsingh
http://www.rajkrrsingh.blogspot.com
Mobile Tel: +91 (0)9899821370
On Sun, May 11, 2014 at 7:27 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by unmesha sreeveni <un...@gmail.com>.
Yes you can do so. Hadoop is a distributed computing framework. And you are
able to do multiple writes also. Only thing we cannot do is we will not be
able to update a file content. But you can do this by deleting and them
writing the whole once again
On Mon, May 12, 2014 at 8:06 AM, Stanley Shi <ss...@gopivotal.com> wrote:
> Yes, why not?
>
> Regards,
> *Stanley Shi,*
>
>
>
> On Sun, May 11, 2014 at 9:57 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
>> Hi,
>>
>> Can I open multiple files on hdfs and write data to them in parallel and
>> then close them at the end?
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>
>
>
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Center for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/
Re: writing multiple files on hdfs
Posted by unmesha sreeveni <un...@gmail.com>.
Yes you can do so. Hadoop is a distributed computing framework. And you are
able to do multiple writes also. Only thing we cannot do is we will not be
able to update a file content. But you can do this by deleting and them
writing the whole once again
On Mon, May 12, 2014 at 8:06 AM, Stanley Shi <ss...@gopivotal.com> wrote:
> Yes, why not?
>
> Regards,
> *Stanley Shi,*
>
>
>
> On Sun, May 11, 2014 at 9:57 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
>> Hi,
>>
>> Can I open multiple files on hdfs and write data to them in parallel and
>> then close them at the end?
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>
>
>
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Center for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/
Re: writing multiple files on hdfs
Posted by unmesha sreeveni <un...@gmail.com>.
Yes you can do so. Hadoop is a distributed computing framework. And you are
able to do multiple writes also. Only thing we cannot do is we will not be
able to update a file content. But you can do this by deleting and them
writing the whole once again
On Mon, May 12, 2014 at 8:06 AM, Stanley Shi <ss...@gopivotal.com> wrote:
> Yes, why not?
>
> Regards,
> *Stanley Shi,*
>
>
>
> On Sun, May 11, 2014 at 9:57 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
>> Hi,
>>
>> Can I open multiple files on hdfs and write data to them in parallel and
>> then close them at the end?
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>
>
>
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Center for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/
Re: writing multiple files on hdfs
Posted by unmesha sreeveni <un...@gmail.com>.
Yes you can do so. Hadoop is a distributed computing framework. And you are
able to do multiple writes also. Only thing we cannot do is we will not be
able to update a file content. But you can do this by deleting and them
writing the whole once again
On Mon, May 12, 2014 at 8:06 AM, Stanley Shi <ss...@gopivotal.com> wrote:
> Yes, why not?
>
> Regards,
> *Stanley Shi,*
>
>
>
> On Sun, May 11, 2014 at 9:57 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
>
>> Hi,
>>
>> Can I open multiple files on hdfs and write data to them in parallel and
>> then close them at the end?
>>
>> --
>> Best Regards,
>> Karim Ahmed Awara
>>
>> ------------------------------
>> This message and its contents, including attachments are intended solely
>> for the original recipient. If you are not the intended recipient or have
>> received this message in error, please notify me immediately and delete
>> this message from your computer system. Any unauthorized use or
>> distribution is prohibited. Please consider the environment before printing
>> this email.
>
>
>
--
*Thanks & Regards *
*Unmesha Sreeveni U.B*
*Hadoop, Bigdata Developer*
*Center for Cyber Security | Amrita Vishwa Vidyapeetham*
http://www.unmeshasreeveni.blogspot.in/
Re: writing multiple files on hdfs
Posted by Stanley Shi <ss...@gopivotal.com>.
Yes, why not?
Regards,
*Stanley Shi,*
On Sun, May 11, 2014 at 9:57 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Raj K Singh <ra...@gmail.com>.
have u tried the hdfs api to create or write a file to the HDFS, will it
preventing u opening multiple OutputStream on the FileSystem?
::::::::::::::::::::::::::::::::::::::::
Raj K Singh
http://in.linkedin.com/in/rajkrrsingh
http://www.rajkrrsingh.blogspot.com
Mobile Tel: +91 (0)9899821370
On Sun, May 11, 2014 at 7:27 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Oleg Zhurakousky <ol...@gmail.com>.
Yes
On Sun, May 11, 2014 at 9:57 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Stanley Shi <ss...@gopivotal.com>.
Yes, why not?
Regards,
*Stanley Shi,*
On Sun, May 11, 2014 at 9:57 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Raj K Singh <ra...@gmail.com>.
have u tried the hdfs api to create or write a file to the HDFS, will it
preventing u opening multiple OutputStream on the FileSystem?
::::::::::::::::::::::::::::::::::::::::
Raj K Singh
http://in.linkedin.com/in/rajkrrsingh
http://www.rajkrrsingh.blogspot.com
Mobile Tel: +91 (0)9899821370
On Sun, May 11, 2014 at 7:27 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Stanley Shi <ss...@gopivotal.com>.
Yes, why not?
Regards,
*Stanley Shi,*
On Sun, May 11, 2014 at 9:57 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Stanley Shi <ss...@gopivotal.com>.
Yes, why not?
Regards,
*Stanley Shi,*
On Sun, May 11, 2014 at 9:57 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Raj K Singh <ra...@gmail.com>.
have u tried the hdfs api to create or write a file to the HDFS, will it
preventing u opening multiple OutputStream on the FileSystem?
::::::::::::::::::::::::::::::::::::::::
Raj K Singh
http://in.linkedin.com/in/rajkrrsingh
http://www.rajkrrsingh.blogspot.com
Mobile Tel: +91 (0)9899821370
On Sun, May 11, 2014 at 7:27 PM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Oleg Zhurakousky <ol...@gmail.com>.
Yes
On Sun, May 11, 2014 at 9:57 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Krishna Chaitanya <bn...@gmail.com>.
Yes!
On May 14, 2014 10:34 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Oleg Zhurakousky <ol...@gmail.com>.
Yes
On Sun, May 11, 2014 at 9:57 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Oleg Zhurakousky <ol...@gmail.com>.
Yes
On Sun, May 11, 2014 at 9:57 AM, Karim Awara <ka...@kaust.edu.sa>wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Krishna Chaitanya <bn...@gmail.com>.
Yes!
On May 14, 2014 10:34 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.
Re: writing multiple files on hdfs
Posted by Krishna Chaitanya <bn...@gmail.com>.
Yes!
On May 14, 2014 10:34 PM, "Karim Awara" <ka...@kaust.edu.sa> wrote:
> Hi,
>
> Can I open multiple files on hdfs and write data to them in parallel and
> then close them at the end?
>
> --
> Best Regards,
> Karim Ahmed Awara
>
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.