[jira] [Created] (HDFS-17069) The documentation and implementation of "dfs.blocksize" are inconsistent.

["ECFuzz (Jira)" <ji...@apache.org>]

ECFuzz created HDFS-17069:
-----------------------------

             Summary: The documentation and implementation of "dfs.blocksize" are inconsistent.
                 Key: HDFS-17069
                 URL: https://issues.apache.org/jira/browse/HDFS-17069
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: dfs, documentation
    Affects Versions: 3.3.6
         Environment: Linux version 4.15.0-142-generic (buildd@lgw01-amd64-039) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.12))

java version "1.8.0_162"
Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
            Reporter: ECFuzz


My hadoop version is 3.3.6, and I use the Pseudo-Distributed Operation.

`core-site.xml` like below.

```shell
<configuration>
  <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
    <property>
        <name>hadoop.tmp.dir</name>
        <value>/home/hadoop/Mutil_Component/tmp</value>
    </property>
   
</configuration>
```

`hdfs-site.xml` like below.

```shell
<configuration>
   <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
<property>
        <name>dfs.blocksize</name>
        <value>128k</value>
    </property>
   
</configuration>
```

 

And then format the namenode, and start the hdfs.

```shell
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$ bin/hdfs namenode -format
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx(many info)
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$ sbin/start-dfs.sh
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [hadoop-Standard-PC-i440FX-PIIX-1996]

```

Finally, use dfs to put a file. Then I get the message which means 128k is less than 1M.

```shell

hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$ bin/hdfs dfs -mkdir -p /user/hadoop
hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$ bin/hdfs dfs -mkdir input

hadoop@hadoop-Standard-PC-i440FX-PIIX-1996:~/Mutil_Component/hadoop-3.3.6$ bin/hdfs dfs -put etc/hadoop/hdfs-site.xml input
put: Specified block size is less than configured minimum value (dfs.namenode.fs-limits.min-block-size): 131072 < 1048576

```

 

But I find that in the document, dfs.blocksize can be set like 128k and other values in `hdfs-default.xml` .

```shell

The default block size for new files, in bytes. You can use the following suffix (case insensitive): k(kilo), m(mega), g(giga), t(tera), p(peta), e(exa) to specify the size (such as 128k, 512m, 1g, etc.), Or provide complete size in bytes (such as 134217728 for 128 MB).

```

So, should there be some issues with the documents here？Or should notice user to set this configuration to be lager than 1M?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org