You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Bao Sheng Li <di...@gmail.com> on 2015/06/15 16:00:49 UTC
Set Replica Issue
Hello,
I'm running Hadoop 2.7.0 on "Red Hat Enterprise Linux Server release 6.5".
There are two problems are pretty annoying:
1. whenever I run hadoop command, I got the warning:
WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
2. The command used to set replicas just hangs there:
$ ./hadoop fs -setrep -w 3 /user/dal/input/capacity-scheduler.xml
15/06/12 16:19:40 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Replication 3 set: /user/dal/input/capacity-scheduler.xml
Waiting for /user/dal/input/capacity-scheduler.xml ....
Any help would be appreciated!
Diamond
Re: Set Replica Issue
Posted by Bao Sheng Li <di...@gmail.com>.
Hi Yanbo,
I run stat to verify that this file has 3 replicas. If this is a shell
problem, any way to work around, otherwise, it just hangs. Also, I check
NameNode log, no any clues.
dal@swat1lxas02 bin]$ ./hdfs dfs -stat "%r"
/user/dal/input/capacity-scheduler.xml
15/06/16 09:32:37 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
3
[dal@swat1lxas02 bin]$ ./hadoop fs -setrep -w 3
/user/dal/input/capacity-scheduler.xml
15/06/16 09:33:01 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Replication 3 set: /user/dal/input/capacity-scheduler.xml
Waiting for /user/dal/input/capacity-scheduler.xml .....
On Tue, Jun 16, 2015 at 12:18 AM, Yanbo Liang <yb...@gmail.com> wrote:
> 1, It means that you can not use native library for your platform which is
> written by C/C++ and will performance benefit. However, it can be replaced
> by buildin-java classes. This is a warning log not error one, so it doesn't
> matter.
> 2, You can check the replicas number of this file by other ways. If it
> satisfy with the replicas restriction, it may be some problems with the
> shell output. If it does not satisfy, try to find clue from logs.
>
> 2015-06-15 22:00 GMT+08:00 Bao Sheng Li <di...@gmail.com>:
>
>> Hello,
>>
>> I'm running Hadoop 2.7.0 on "Red Hat Enterprise Linux Server release
>> 6.5". There are two problems are pretty annoying:
>>
>> 1. whenever I run hadoop command, I got the warning:
>> WARN util.NativeCodeLoader: Unable to load native-hadoop library for
>> your platform... using builtin-java classes where applicable
>>
>> 2. The command used to set replicas just hangs there:
>>
>> $ ./hadoop fs -setrep -w 3 /user/dal/input/capacity-scheduler.xml
>> 15/06/12 16:19:40 WARN util.NativeCodeLoader: Unable to load
>> native-hadoop library for your platform... using builtin-java classes where
>> applicable
>> Replication 3 set: /user/dal/input/capacity-scheduler.xml
>> Waiting for /user/dal/input/capacity-scheduler.xml ....
>>
>> Any help would be appreciated!
>>
>> Diamond
>>
>
>
Re: Set Replica Issue
Posted by Bao Sheng Li <di...@gmail.com>.
Hi Yanbo,
I run stat to verify that this file has 3 replicas. If this is a shell
problem, any way to work around, otherwise, it just hangs. Also, I check
NameNode log, no any clues.
dal@swat1lxas02 bin]$ ./hdfs dfs -stat "%r"
/user/dal/input/capacity-scheduler.xml
15/06/16 09:32:37 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
3
[dal@swat1lxas02 bin]$ ./hadoop fs -setrep -w 3
/user/dal/input/capacity-scheduler.xml
15/06/16 09:33:01 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Replication 3 set: /user/dal/input/capacity-scheduler.xml
Waiting for /user/dal/input/capacity-scheduler.xml .....
On Tue, Jun 16, 2015 at 12:18 AM, Yanbo Liang <yb...@gmail.com> wrote:
> 1, It means that you can not use native library for your platform which is
> written by C/C++ and will performance benefit. However, it can be replaced
> by buildin-java classes. This is a warning log not error one, so it doesn't
> matter.
> 2, You can check the replicas number of this file by other ways. If it
> satisfy with the replicas restriction, it may be some problems with the
> shell output. If it does not satisfy, try to find clue from logs.
>
> 2015-06-15 22:00 GMT+08:00 Bao Sheng Li <di...@gmail.com>:
>
>> Hello,
>>
>> I'm running Hadoop 2.7.0 on "Red Hat Enterprise Linux Server release
>> 6.5". There are two problems are pretty annoying:
>>
>> 1. whenever I run hadoop command, I got the warning:
>> WARN util.NativeCodeLoader: Unable to load native-hadoop library for
>> your platform... using builtin-java classes where applicable
>>
>> 2. The command used to set replicas just hangs there:
>>
>> $ ./hadoop fs -setrep -w 3 /user/dal/input/capacity-scheduler.xml
>> 15/06/12 16:19:40 WARN util.NativeCodeLoader: Unable to load
>> native-hadoop library for your platform... using builtin-java classes where
>> applicable
>> Replication 3 set: /user/dal/input/capacity-scheduler.xml
>> Waiting for /user/dal/input/capacity-scheduler.xml ....
>>
>> Any help would be appreciated!
>>
>> Diamond
>>
>
>
Re: Set Replica Issue
Posted by Bao Sheng Li <di...@gmail.com>.
Hi Yanbo,
I run stat to verify that this file has 3 replicas. If this is a shell
problem, any way to work around, otherwise, it just hangs. Also, I check
NameNode log, no any clues.
dal@swat1lxas02 bin]$ ./hdfs dfs -stat "%r"
/user/dal/input/capacity-scheduler.xml
15/06/16 09:32:37 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
3
[dal@swat1lxas02 bin]$ ./hadoop fs -setrep -w 3
/user/dal/input/capacity-scheduler.xml
15/06/16 09:33:01 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Replication 3 set: /user/dal/input/capacity-scheduler.xml
Waiting for /user/dal/input/capacity-scheduler.xml .....
On Tue, Jun 16, 2015 at 12:18 AM, Yanbo Liang <yb...@gmail.com> wrote:
> 1, It means that you can not use native library for your platform which is
> written by C/C++ and will performance benefit. However, it can be replaced
> by buildin-java classes. This is a warning log not error one, so it doesn't
> matter.
> 2, You can check the replicas number of this file by other ways. If it
> satisfy with the replicas restriction, it may be some problems with the
> shell output. If it does not satisfy, try to find clue from logs.
>
> 2015-06-15 22:00 GMT+08:00 Bao Sheng Li <di...@gmail.com>:
>
>> Hello,
>>
>> I'm running Hadoop 2.7.0 on "Red Hat Enterprise Linux Server release
>> 6.5". There are two problems are pretty annoying:
>>
>> 1. whenever I run hadoop command, I got the warning:
>> WARN util.NativeCodeLoader: Unable to load native-hadoop library for
>> your platform... using builtin-java classes where applicable
>>
>> 2. The command used to set replicas just hangs there:
>>
>> $ ./hadoop fs -setrep -w 3 /user/dal/input/capacity-scheduler.xml
>> 15/06/12 16:19:40 WARN util.NativeCodeLoader: Unable to load
>> native-hadoop library for your platform... using builtin-java classes where
>> applicable
>> Replication 3 set: /user/dal/input/capacity-scheduler.xml
>> Waiting for /user/dal/input/capacity-scheduler.xml ....
>>
>> Any help would be appreciated!
>>
>> Diamond
>>
>
>
Re: Set Replica Issue
Posted by Bao Sheng Li <di...@gmail.com>.
Hi Yanbo,
I run stat to verify that this file has 3 replicas. If this is a shell
problem, any way to work around, otherwise, it just hangs. Also, I check
NameNode log, no any clues.
dal@swat1lxas02 bin]$ ./hdfs dfs -stat "%r"
/user/dal/input/capacity-scheduler.xml
15/06/16 09:32:37 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
3
[dal@swat1lxas02 bin]$ ./hadoop fs -setrep -w 3
/user/dal/input/capacity-scheduler.xml
15/06/16 09:33:01 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Replication 3 set: /user/dal/input/capacity-scheduler.xml
Waiting for /user/dal/input/capacity-scheduler.xml .....
On Tue, Jun 16, 2015 at 12:18 AM, Yanbo Liang <yb...@gmail.com> wrote:
> 1, It means that you can not use native library for your platform which is
> written by C/C++ and will performance benefit. However, it can be replaced
> by buildin-java classes. This is a warning log not error one, so it doesn't
> matter.
> 2, You can check the replicas number of this file by other ways. If it
> satisfy with the replicas restriction, it may be some problems with the
> shell output. If it does not satisfy, try to find clue from logs.
>
> 2015-06-15 22:00 GMT+08:00 Bao Sheng Li <di...@gmail.com>:
>
>> Hello,
>>
>> I'm running Hadoop 2.7.0 on "Red Hat Enterprise Linux Server release
>> 6.5". There are two problems are pretty annoying:
>>
>> 1. whenever I run hadoop command, I got the warning:
>> WARN util.NativeCodeLoader: Unable to load native-hadoop library for
>> your platform... using builtin-java classes where applicable
>>
>> 2. The command used to set replicas just hangs there:
>>
>> $ ./hadoop fs -setrep -w 3 /user/dal/input/capacity-scheduler.xml
>> 15/06/12 16:19:40 WARN util.NativeCodeLoader: Unable to load
>> native-hadoop library for your platform... using builtin-java classes where
>> applicable
>> Replication 3 set: /user/dal/input/capacity-scheduler.xml
>> Waiting for /user/dal/input/capacity-scheduler.xml ....
>>
>> Any help would be appreciated!
>>
>> Diamond
>>
>
>
Re: Set Replica Issue
Posted by Yanbo Liang <yb...@gmail.com>.
1, It means that you can not use native library for your platform which is
written by C/C++ and will performance benefit. However, it can be replaced
by buildin-java classes. This is a warning log not error one, so it doesn't
matter.
2, You can check the replicas number of this file by other ways. If it
satisfy with the replicas restriction, it may be some problems with the
shell output. If it does not satisfy, try to find clue from logs.
2015-06-15 22:00 GMT+08:00 Bao Sheng Li <di...@gmail.com>:
> Hello,
>
> I'm running Hadoop 2.7.0 on "Red Hat Enterprise Linux Server release 6.5".
> There are two problems are pretty annoying:
>
> 1. whenever I run hadoop command, I got the warning:
> WARN util.NativeCodeLoader: Unable to load native-hadoop library for
> your platform... using builtin-java classes where applicable
>
> 2. The command used to set replicas just hangs there:
>
> $ ./hadoop fs -setrep -w 3 /user/dal/input/capacity-scheduler.xml
> 15/06/12 16:19:40 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Replication 3 set: /user/dal/input/capacity-scheduler.xml
> Waiting for /user/dal/input/capacity-scheduler.xml ....
>
> Any help would be appreciated!
>
> Diamond
>
Re: Set Replica Issue
Posted by Yanbo Liang <yb...@gmail.com>.
1, It means that you can not use native library for your platform which is
written by C/C++ and will performance benefit. However, it can be replaced
by buildin-java classes. This is a warning log not error one, so it doesn't
matter.
2, You can check the replicas number of this file by other ways. If it
satisfy with the replicas restriction, it may be some problems with the
shell output. If it does not satisfy, try to find clue from logs.
2015-06-15 22:00 GMT+08:00 Bao Sheng Li <di...@gmail.com>:
> Hello,
>
> I'm running Hadoop 2.7.0 on "Red Hat Enterprise Linux Server release 6.5".
> There are two problems are pretty annoying:
>
> 1. whenever I run hadoop command, I got the warning:
> WARN util.NativeCodeLoader: Unable to load native-hadoop library for
> your platform... using builtin-java classes where applicable
>
> 2. The command used to set replicas just hangs there:
>
> $ ./hadoop fs -setrep -w 3 /user/dal/input/capacity-scheduler.xml
> 15/06/12 16:19:40 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Replication 3 set: /user/dal/input/capacity-scheduler.xml
> Waiting for /user/dal/input/capacity-scheduler.xml ....
>
> Any help would be appreciated!
>
> Diamond
>
Re: Set Replica Issue
Posted by Yanbo Liang <yb...@gmail.com>.
1, It means that you can not use native library for your platform which is
written by C/C++ and will performance benefit. However, it can be replaced
by buildin-java classes. This is a warning log not error one, so it doesn't
matter.
2, You can check the replicas number of this file by other ways. If it
satisfy with the replicas restriction, it may be some problems with the
shell output. If it does not satisfy, try to find clue from logs.
2015-06-15 22:00 GMT+08:00 Bao Sheng Li <di...@gmail.com>:
> Hello,
>
> I'm running Hadoop 2.7.0 on "Red Hat Enterprise Linux Server release 6.5".
> There are two problems are pretty annoying:
>
> 1. whenever I run hadoop command, I got the warning:
> WARN util.NativeCodeLoader: Unable to load native-hadoop library for
> your platform... using builtin-java classes where applicable
>
> 2. The command used to set replicas just hangs there:
>
> $ ./hadoop fs -setrep -w 3 /user/dal/input/capacity-scheduler.xml
> 15/06/12 16:19:40 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Replication 3 set: /user/dal/input/capacity-scheduler.xml
> Waiting for /user/dal/input/capacity-scheduler.xml ....
>
> Any help would be appreciated!
>
> Diamond
>
Re: Set Replica Issue
Posted by Yanbo Liang <yb...@gmail.com>.
1, It means that you can not use native library for your platform which is
written by C/C++ and will performance benefit. However, it can be replaced
by buildin-java classes. This is a warning log not error one, so it doesn't
matter.
2, You can check the replicas number of this file by other ways. If it
satisfy with the replicas restriction, it may be some problems with the
shell output. If it does not satisfy, try to find clue from logs.
2015-06-15 22:00 GMT+08:00 Bao Sheng Li <di...@gmail.com>:
> Hello,
>
> I'm running Hadoop 2.7.0 on "Red Hat Enterprise Linux Server release 6.5".
> There are two problems are pretty annoying:
>
> 1. whenever I run hadoop command, I got the warning:
> WARN util.NativeCodeLoader: Unable to load native-hadoop library for
> your platform... using builtin-java classes where applicable
>
> 2. The command used to set replicas just hangs there:
>
> $ ./hadoop fs -setrep -w 3 /user/dal/input/capacity-scheduler.xml
> 15/06/12 16:19:40 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Replication 3 set: /user/dal/input/capacity-scheduler.xml
> Waiting for /user/dal/input/capacity-scheduler.xml ....
>
> Any help would be appreciated!
>
> Diamond
>