You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Jason Rutherglen <ja...@gmail.com> on 2011/04/12 15:36:34 UTC

Setting a larger block size at runtime in the DFSClient

Are there performance implications to setting the block size to 1 GB
or higher (via the DFSClient.create method)?

Re: Setting a larger block size at runtime in the DFSClient

Posted by Jason Rutherglen <ja...@gmail.com>.

Harsh, thanks, and sounds good!

On Tue, Apr 12, 2011 at 7:08 AM, Harsh J <ha...@cloudera.com> wrote:
> Hey Jason,
>
> On Tue, Apr 12, 2011 at 7:06 PM, Jason Rutherglen
> <ja...@gmail.com> wrote:
>> Are there performance implications to setting the block size to 1 GB
>> or higher (via the DFSClient.create method)?
>
> You'll be streaming 1 complete GB per block to a DN with that value
> (before the next block gets scheduled on another), shouldn't be any
> differences beyond that.
>
> --
> Harsh J
>

Re: Setting a larger block size at runtime in the DFSClient

Posted by Harsh J <ha...@cloudera.com>.

Hey Jason,

On Tue, Apr 12, 2011 at 7:06 PM, Jason Rutherglen
<ja...@gmail.com> wrote:
> Are there performance implications to setting the block size to 1 GB
> or higher (via the DFSClient.create method)?

You'll be streaming 1 complete GB per block to a DN with that value
(before the next block gets scheduled on another), shouldn't be any
differences beyond that.

-- 
Harsh J