You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "zhuweimin (JIRA)" <ji...@apache.org> on 2008/12/16 06:38:44 UTC
[jira] Created: (HADOOP-4877) parameter dfs.replication is not
reflected when put file into hadoop with fuse-dfs
parameter dfs.replication is not reflected when put file into hadoop with fuse-dfs
----------------------------------------------------------------------------------
Key: HADOOP-4877
URL: https://issues.apache.org/jira/browse/HADOOP-4877
Project: Hadoop Core
Issue Type: Bug
Components: contrib/fuse-dfs
Affects Versions: 0.19.0
Environment: os:centos5.2
cpu:amd64
hadoop0.19.0
Reporter: zhuweimin
the $HADOOP_CONF_DIR is exist in the $CLASSPATH
and the dfs.replication is set to 3 with the following in hadoop-site.xml
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
The file's replication is 3 when it be put into hadoop.
I think the reason is :
there is a hardcoding in the src\contrib\fuse-dfs\src\fuse_dfs.c
line 1337
if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 3, 0)) == NULL) {
line 1591
if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 3, 0)) == NULL) {
the fifth parameter is a hardcoding when call the function of hdfsOpenFile.
It is should set to 0. It is as follows.
line 1337
if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 0, 0)) == NULL) {
line 1591
if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 0, 0)) == NULL) {
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4877) parameter dfs.replication is not
reflected when put file into hadoop with fuse-dfs
Posted by "zhuweimin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhuweimin updated HADOOP-4877:
------------------------------
Attachment: HADOOP-4877.txt.0.19
> parameter dfs.replication is not reflected when put file into hadoop with fuse-dfs
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-4877
> URL: https://issues.apache.org/jira/browse/HADOOP-4877
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fuse-dfs
> Affects Versions: 0.19.0
> Environment: os:centos5.2
> cpu:amd64
> hadoop0.19.0
> Reporter: zhuweimin
> Attachments: HADOOP-4877.txt.0.19
>
>
> the $HADOOP_CONF_DIR is exist in the $CLASSPATH
> and the dfs.replication is set to 3 with the following in hadoop-site.xml
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> The file's replication is 3 when it be put into hadoop.
> I think the reason is :
> there is a hardcoding in the src\contrib\fuse-dfs\src\fuse_dfs.c
> line 1337
> if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 3, 0)) == NULL) {
> line 1591
> if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 3, 0)) == NULL) {
> the fifth parameter is a hardcoding when call the function of hdfsOpenFile.
> It is should set to 0. It is as follows.
> line 1337
> if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 0, 0)) == NULL) {
> line 1591
> if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 0, 0)) == NULL) {
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4877) parameter dfs.replication is not
reflected when put file into hadoop with fuse-dfs
Posted by "zhuweimin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhuweimin updated HADOOP-4877:
------------------------------
Status: Patch Available (was: Open)
> parameter dfs.replication is not reflected when put file into hadoop with fuse-dfs
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-4877
> URL: https://issues.apache.org/jira/browse/HADOOP-4877
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fuse-dfs
> Affects Versions: 0.19.0
> Environment: os:centos5.2
> cpu:amd64
> hadoop0.19.0
> Reporter: zhuweimin
>
> the $HADOOP_CONF_DIR is exist in the $CLASSPATH
> and the dfs.replication is set to 3 with the following in hadoop-site.xml
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> The file's replication is 3 when it be put into hadoop.
> I think the reason is :
> there is a hardcoding in the src\contrib\fuse-dfs\src\fuse_dfs.c
> line 1337
> if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 3, 0)) == NULL) {
> line 1591
> if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 3, 0)) == NULL) {
> the fifth parameter is a hardcoding when call the function of hdfsOpenFile.
> It is should set to 0. It is as follows.
> line 1337
> if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 0, 0)) == NULL) {
> line 1591
> if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 0, 0)) == NULL) {
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4877) parameter dfs.replication is not
reflected when put file into hadoop with fuse-dfs
Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Macdonald updated HADOOP-4877:
------------------------------------
Attachment: HADOOP-4877.txt.trunk
Here is a patch for trunk. My only thoughts are that for truncate, it might be more uniform if the original file replication number was kept.
> parameter dfs.replication is not reflected when put file into hadoop with fuse-dfs
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-4877
> URL: https://issues.apache.org/jira/browse/HADOOP-4877
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fuse-dfs
> Affects Versions: 0.19.0
> Environment: os:centos5.2
> cpu:amd64
> hadoop0.19.0
> Reporter: zhuweimin
> Attachments: HADOOP-4877.txt.0.19, HADOOP-4877.txt.trunk
>
>
> the $HADOOP_CONF_DIR is exist in the $CLASSPATH
> and the dfs.replication is set to 3 with the following in hadoop-site.xml
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> The file's replication is 3 when it be put into hadoop.
> I think the reason is :
> there is a hardcoding in the src\contrib\fuse-dfs\src\fuse_dfs.c
> line 1337
> if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 3, 0)) == NULL) {
> line 1591
> if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 3, 0)) == NULL) {
> the fifth parameter is a hardcoding when call the function of hdfsOpenFile.
> It is should set to 0. It is as follows.
> line 1337
> if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 0, 0)) == NULL) {
> line 1591
> if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 0, 0)) == NULL) {
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4877) parameter dfs.replication is not
reflected when put file into hadoop with fuse-dfs
Posted by "zhuweimin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhuweimin updated HADOOP-4877:
------------------------------
Status: Open (was: Patch Available)
> parameter dfs.replication is not reflected when put file into hadoop with fuse-dfs
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-4877
> URL: https://issues.apache.org/jira/browse/HADOOP-4877
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fuse-dfs
> Affects Versions: 0.19.0
> Environment: os:centos5.2
> cpu:amd64
> hadoop0.19.0
> Reporter: zhuweimin
>
> the $HADOOP_CONF_DIR is exist in the $CLASSPATH
> and the dfs.replication is set to 3 with the following in hadoop-site.xml
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> The file's replication is 3 when it be put into hadoop.
> I think the reason is :
> there is a hardcoding in the src\contrib\fuse-dfs\src\fuse_dfs.c
> line 1337
> if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 3, 0)) == NULL) {
> line 1591
> if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 3, 0)) == NULL) {
> the fifth parameter is a hardcoding when call the function of hdfsOpenFile.
> It is should set to 0. It is as follows.
> line 1337
> if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 0, 0)) == NULL) {
> line 1591
> if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 0, 0)) == NULL) {
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4877) parameter dfs.replication is not
reflected when put file into hadoop with fuse-dfs
Posted by "zhuweimin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhuweimin updated HADOOP-4877:
------------------------------
Status: Patch Available (was: Open)
> parameter dfs.replication is not reflected when put file into hadoop with fuse-dfs
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-4877
> URL: https://issues.apache.org/jira/browse/HADOOP-4877
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fuse-dfs
> Affects Versions: 0.19.0
> Environment: os:centos5.2
> cpu:amd64
> hadoop0.19.0
> Reporter: zhuweimin
>
> the $HADOOP_CONF_DIR is exist in the $CLASSPATH
> and the dfs.replication is set to 3 with the following in hadoop-site.xml
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> The file's replication is 3 when it be put into hadoop.
> I think the reason is :
> there is a hardcoding in the src\contrib\fuse-dfs\src\fuse_dfs.c
> line 1337
> if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 3, 0)) == NULL) {
> line 1591
> if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 3, 0)) == NULL) {
> the fifth parameter is a hardcoding when call the function of hdfsOpenFile.
> It is should set to 0. It is as follows.
> line 1337
> if ((fh->hdfsFH = (hdfsFile)hdfsOpenFile(fh->fs, path, flags, 0, 0, 0)) == NULL) {
> line 1591
> if ((file = (hdfsFile)hdfsOpenFile(userFS, path, flags, 0, 0, 0)) == NULL) {
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.