You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Harsh J <ha...@cloudera.com> on 2011/11/19 07:50:06 UTC
Re: How many files can a hdfs client can access simultaneously
Kartheek,
(Moving to hdfs-user@. Bcc'd common-user@. Please use the right lists for reaching to the proper audience :))
Writing multiple files from a single DFS client instance should not be a problem at all. You get one output stream per file creation, leaving you're free to write to a number of them in parallel and to also expect an amount of gain.
On 19-Nov-2011, at 11:26 AM, kartheek muthyala wrote:
> Hi all,
> I know that while writing the file, a hdfs client writes the blocks in
> sequential fashion, writing one block only at one time and when this block
> is acknowledged as a complete write then the next block write happens. But
> can a hdfs client write two or three files at a same time, ideally it
> should do, can i get a confirmation on this?
> ~Kartheek
Re: How many files can a hdfs client can access simultaneously
Posted by Harsh J <ha...@cloudera.com>.
Oops, s/you're/you. I rephrased half-way.
On 19-Nov-2011, at 2:02 PM, kartheek muthyala wrote:
> Thanks harsh for the confirmation and for referring me to this dl. :)
>
> On Sat, Nov 19, 2011 at 12:20 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Kartheek,
>>
>> (Moving to hdfs-user@. Bcc'd common-user@. Please use the right lists for
>> reaching to the proper audience :))
>>
>> Writing multiple files from a single DFS client instance should not be a
>> problem at all. You get one output stream per file creation, leaving you're
>> free to write to a number of them in parallel and to also expect an amount
>> of gain.
>>
>> On 19-Nov-2011, at 11:26 AM, kartheek muthyala wrote:
>>
>>> Hi all,
>>> I know that while writing the file, a hdfs client writes the blocks in
>>> sequential fashion, writing one block only at one time and when this
>> block
>>> is acknowledged as a complete write then the next block write happens.
>> But
>>> can a hdfs client write two or three files at a same time, ideally it
>>> should do, can i get a confirmation on this?
>>> ~Kartheek
>>
>>
Re: How many files can a hdfs client can access simultaneously
Posted by Harsh J <ha...@cloudera.com>.
Oops, s/you're/you. I rephrased half-way.
On 19-Nov-2011, at 2:02 PM, kartheek muthyala wrote:
> Thanks harsh for the confirmation and for referring me to this dl. :)
>
> On Sat, Nov 19, 2011 at 12:20 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Kartheek,
>>
>> (Moving to hdfs-user@. Bcc'd common-user@. Please use the right lists for
>> reaching to the proper audience :))
>>
>> Writing multiple files from a single DFS client instance should not be a
>> problem at all. You get one output stream per file creation, leaving you're
>> free to write to a number of them in parallel and to also expect an amount
>> of gain.
>>
>> On 19-Nov-2011, at 11:26 AM, kartheek muthyala wrote:
>>
>>> Hi all,
>>> I know that while writing the file, a hdfs client writes the blocks in
>>> sequential fashion, writing one block only at one time and when this
>> block
>>> is acknowledged as a complete write then the next block write happens.
>> But
>>> can a hdfs client write two or three files at a same time, ideally it
>>> should do, can i get a confirmation on this?
>>> ~Kartheek
>>
>>
Re: How many files can a hdfs client can access simultaneously
Posted by kartheek muthyala <ka...@gmail.com>.
Thanks harsh for the confirmation and for referring me to this dl. :)
On Sat, Nov 19, 2011 at 12:20 PM, Harsh J <ha...@cloudera.com> wrote:
> Kartheek,
>
> (Moving to hdfs-user@. Bcc'd common-user@. Please use the right lists for
> reaching to the proper audience :))
>
> Writing multiple files from a single DFS client instance should not be a
> problem at all. You get one output stream per file creation, leaving you're
> free to write to a number of them in parallel and to also expect an amount
> of gain.
>
> On 19-Nov-2011, at 11:26 AM, kartheek muthyala wrote:
>
> > Hi all,
> > I know that while writing the file, a hdfs client writes the blocks in
> > sequential fashion, writing one block only at one time and when this
> block
> > is acknowledged as a complete write then the next block write happens.
> But
> > can a hdfs client write two or three files at a same time, ideally it
> > should do, can i get a confirmation on this?
> > ~Kartheek
>
>