You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Satyam Singh <sa...@ericsson.com> on 2014/07/29 09:55:43 UTC
Which replication value will be used
Hello,
I have given dfs.replication=2 in hdfs-site.xml as:
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
Also, In my application i have used api for wrting in hdfs:
public FSDataOutputStream create(Path f,
boolean overwrite,
int bufferSize,
short replication,
long blockSize,
Progressable progress
) throws IOException {
return this.create(f, FsPermission.getFileDefault().applyUMask(
FsPermission.getUMask(getConf())), overwrite, bufferSize,
replication, blockSize, progress);
}
here i also given replication as fourth parameter.
So my question is which replication value will be used that we have
given in api or in hdfs-site.xml
Warm Regards,
Satyam
Re: Which replication value will be used
Posted by Nitin Pawar <ni...@gmail.com>.
first it will check if you have set replication factor in your write call
.. if yes .. it will be respected.
if there is no value for replication in write process, it falls back to
default value in hdfs-site.xml
On Tue, Jul 29, 2014 at 1:25 PM, Satyam Singh <sa...@ericsson.com>
wrote:
> Hello,
>
>
>
> I have given dfs.replication=2 in hdfs-site.xml as:
> <property>
> <name>dfs.replication</name>
> <value>2</value>
> </property>
>
> Also, In my application i have used api for wrting in hdfs:
> public FSDataOutputStream create(Path f,
> boolean overwrite,
> int bufferSize,
> short replication,
> long blockSize,
> Progressable progress
> ) throws IOException {
> return this.create(f, FsPermission.getFileDefault().applyUMask(
> FsPermission.getUMask(getConf())), overwrite, bufferSize,
> replication, blockSize, progress);
> }
>
> here i also given replication as fourth parameter.
>
> So my question is which replication value will be used that we have given
> in api or in hdfs-site.xml
>
>
> Warm Regards,
> Satyam
>
--
Nitin Pawar
Re: Which replication value will be used
Posted by hadoop hive <ha...@gmail.com>.
Its should pick whatever value you are providing in your api only If in
hdfs-site.XML doesn't have set final in replication parameter.
On Jul 29, 2014 1:26 PM, "Satyam Singh" <sa...@ericsson.com> wrote:
> Hello,
>
>
>
> I have given dfs.replication=2 in hdfs-site.xml as:
> <property>
> <name>dfs.replication</name>
> <value>2</value>
> </property>
>
> Also, In my application i have used api for wrting in hdfs:
> public FSDataOutputStream create(Path f,
> boolean overwrite,
> int bufferSize,
> short replication,
> long blockSize,
> Progressable progress
> ) throws IOException {
> return this.create(f, FsPermission.getFileDefault().applyUMask(
> FsPermission.getUMask(getConf())), overwrite, bufferSize,
> replication, blockSize, progress);
> }
>
> here i also given replication as fourth parameter.
>
> So my question is which replication value will be used that we have given
> in api or in hdfs-site.xml
>
>
> Warm Regards,
> Satyam
>
Re: Which replication value will be used
Posted by Nitin Pawar <ni...@gmail.com>.
first it will check if you have set replication factor in your write call
.. if yes .. it will be respected.
if there is no value for replication in write process, it falls back to
default value in hdfs-site.xml
On Tue, Jul 29, 2014 at 1:25 PM, Satyam Singh <sa...@ericsson.com>
wrote:
> Hello,
>
>
>
> I have given dfs.replication=2 in hdfs-site.xml as:
> <property>
> <name>dfs.replication</name>
> <value>2</value>
> </property>
>
> Also, In my application i have used api for wrting in hdfs:
> public FSDataOutputStream create(Path f,
> boolean overwrite,
> int bufferSize,
> short replication,
> long blockSize,
> Progressable progress
> ) throws IOException {
> return this.create(f, FsPermission.getFileDefault().applyUMask(
> FsPermission.getUMask(getConf())), overwrite, bufferSize,
> replication, blockSize, progress);
> }
>
> here i also given replication as fourth parameter.
>
> So my question is which replication value will be used that we have given
> in api or in hdfs-site.xml
>
>
> Warm Regards,
> Satyam
>
--
Nitin Pawar
Re: Which replication value will be used
Posted by Nitin Pawar <ni...@gmail.com>.
first it will check if you have set replication factor in your write call
.. if yes .. it will be respected.
if there is no value for replication in write process, it falls back to
default value in hdfs-site.xml
On Tue, Jul 29, 2014 at 1:25 PM, Satyam Singh <sa...@ericsson.com>
wrote:
> Hello,
>
>
>
> I have given dfs.replication=2 in hdfs-site.xml as:
> <property>
> <name>dfs.replication</name>
> <value>2</value>
> </property>
>
> Also, In my application i have used api for wrting in hdfs:
> public FSDataOutputStream create(Path f,
> boolean overwrite,
> int bufferSize,
> short replication,
> long blockSize,
> Progressable progress
> ) throws IOException {
> return this.create(f, FsPermission.getFileDefault().applyUMask(
> FsPermission.getUMask(getConf())), overwrite, bufferSize,
> replication, blockSize, progress);
> }
>
> here i also given replication as fourth parameter.
>
> So my question is which replication value will be used that we have given
> in api or in hdfs-site.xml
>
>
> Warm Regards,
> Satyam
>
--
Nitin Pawar
Re: Which replication value will be used
Posted by hadoop hive <ha...@gmail.com>.
Its should pick whatever value you are providing in your api only If in
hdfs-site.XML doesn't have set final in replication parameter.
On Jul 29, 2014 1:26 PM, "Satyam Singh" <sa...@ericsson.com> wrote:
> Hello,
>
>
>
> I have given dfs.replication=2 in hdfs-site.xml as:
> <property>
> <name>dfs.replication</name>
> <value>2</value>
> </property>
>
> Also, In my application i have used api for wrting in hdfs:
> public FSDataOutputStream create(Path f,
> boolean overwrite,
> int bufferSize,
> short replication,
> long blockSize,
> Progressable progress
> ) throws IOException {
> return this.create(f, FsPermission.getFileDefault().applyUMask(
> FsPermission.getUMask(getConf())), overwrite, bufferSize,
> replication, blockSize, progress);
> }
>
> here i also given replication as fourth parameter.
>
> So my question is which replication value will be used that we have given
> in api or in hdfs-site.xml
>
>
> Warm Regards,
> Satyam
>
Re: Which replication value will be used
Posted by hadoop hive <ha...@gmail.com>.
Its should pick whatever value you are providing in your api only If in
hdfs-site.XML doesn't have set final in replication parameter.
On Jul 29, 2014 1:26 PM, "Satyam Singh" <sa...@ericsson.com> wrote:
> Hello,
>
>
>
> I have given dfs.replication=2 in hdfs-site.xml as:
> <property>
> <name>dfs.replication</name>
> <value>2</value>
> </property>
>
> Also, In my application i have used api for wrting in hdfs:
> public FSDataOutputStream create(Path f,
> boolean overwrite,
> int bufferSize,
> short replication,
> long blockSize,
> Progressable progress
> ) throws IOException {
> return this.create(f, FsPermission.getFileDefault().applyUMask(
> FsPermission.getUMask(getConf())), overwrite, bufferSize,
> replication, blockSize, progress);
> }
>
> here i also given replication as fourth parameter.
>
> So my question is which replication value will be used that we have given
> in api or in hdfs-site.xml
>
>
> Warm Regards,
> Satyam
>
Re: Which replication value will be used
Posted by hadoop hive <ha...@gmail.com>.
Its should pick whatever value you are providing in your api only If in
hdfs-site.XML doesn't have set final in replication parameter.
On Jul 29, 2014 1:26 PM, "Satyam Singh" <sa...@ericsson.com> wrote:
> Hello,
>
>
>
> I have given dfs.replication=2 in hdfs-site.xml as:
> <property>
> <name>dfs.replication</name>
> <value>2</value>
> </property>
>
> Also, In my application i have used api for wrting in hdfs:
> public FSDataOutputStream create(Path f,
> boolean overwrite,
> int bufferSize,
> short replication,
> long blockSize,
> Progressable progress
> ) throws IOException {
> return this.create(f, FsPermission.getFileDefault().applyUMask(
> FsPermission.getUMask(getConf())), overwrite, bufferSize,
> replication, blockSize, progress);
> }
>
> here i also given replication as fourth parameter.
>
> So my question is which replication value will be used that we have given
> in api or in hdfs-site.xml
>
>
> Warm Regards,
> Satyam
>
Re: Which replication value will be used
Posted by Nitin Pawar <ni...@gmail.com>.
first it will check if you have set replication factor in your write call
.. if yes .. it will be respected.
if there is no value for replication in write process, it falls back to
default value in hdfs-site.xml
On Tue, Jul 29, 2014 at 1:25 PM, Satyam Singh <sa...@ericsson.com>
wrote:
> Hello,
>
>
>
> I have given dfs.replication=2 in hdfs-site.xml as:
> <property>
> <name>dfs.replication</name>
> <value>2</value>
> </property>
>
> Also, In my application i have used api for wrting in hdfs:
> public FSDataOutputStream create(Path f,
> boolean overwrite,
> int bufferSize,
> short replication,
> long blockSize,
> Progressable progress
> ) throws IOException {
> return this.create(f, FsPermission.getFileDefault().applyUMask(
> FsPermission.getUMask(getConf())), overwrite, bufferSize,
> replication, blockSize, progress);
> }
>
> here i also given replication as fourth parameter.
>
> So my question is which replication value will be used that we have given
> in api or in hdfs-site.xml
>
>
> Warm Regards,
> Satyam
>
--
Nitin Pawar