You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by xeonmailinglist <xe...@gmail.com> on 2015/03/03 11:20:22 UTC
Can't list files in a federation of HDFS
Hi,
I have configured in 2 hosts (hadooop-coc-1, and hadoop-coc-2) a
federation of HDFS. In the configuration, I have set a namespace in each
host, and a single data node (see image). The service is running
properly. You can check the output of the |jps| commands in [1].
The strange part is that, when I list the files, they do not appear in
hadoop-coc-2. You can check the output in [2]. Why this happens?
hdfsfederation
[1]: jps output
|xubuntu@hadoop-coc-1:~/Programs/hadoop$ jps
21538 NameNode
21773 DataNode
xubuntu@hadoop-coc-2:~/Programs/hadoop$ jps
2342 NameNode
|
[2]: hdfs dfs -ls / output
|xubuntu@hadoop-coc-1:~/Programs/hadoop$ hdfs dfs -ls /
Java HotSpot(TM) Client VM warning: You have loaded library /home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
15/03/03 05:09:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 1 items
drwxr-xr-x - xubuntu supergroup 0 2015-03-03 04:47 /input1
xubuntu@hadoop-coc-2:~/Programs/hadoop$ hdfs dfs -ls /
Java HotSpot(TM) Client VM warning: You have loaded library /home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
15/03/03 05:09:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
|
--
--
Re: Can't list files in a federation of HDFS
Posted by Azuryy Yu <az...@gmail.com>.
For HDFS federation, data share all datanodes, but namespace is separate,
so did you write some data on hadoop-coc-2 namespace? you don't need to
login hadoop-coc-2 then write data, just config a new client, which connect
to hadoop-coc-2 for write.
On Tue, Mar 3, 2015 at 6:20 PM, xeonmailinglist <xe...@gmail.com>
wrote:
> Hi,
>
> I have configured in 2 hosts (hadooop-coc-1, and hadoop-coc-2) a
> federation of HDFS. In the configuration, I have set a namespace in each
> host, and a single data node (see image). The service is running properly.
> You can check the output of the jps commands in [1].
>
> The strange part is that, when I list the files, they do not appear in
> hadoop-coc-2. You can check the output in [2]. Why this happens?
>
> [image: hdfsfederation]
>
> [1]: jps output
>
> xubuntu@hadoop-coc-1:~/Programs/hadoop$ jps
> 21538 NameNode
> 21773 DataNode
>
> xubuntu@hadoop-coc-2:~/Programs/hadoop$ jps
> 2342 NameNode
>
> [2]: hdfs dfs -ls / output
>
> xubuntu@hadoop-coc-1:~/Programs/hadoop$ hdfs dfs -ls /
> Java HotSpot(TM) Client VM warning: You have loaded library /home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
> It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
> 15/03/03 05:09:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> Found 1 items
> drwxr-xr-x - xubuntu supergroup 0 2015-03-03 04:47 /input1
>
> xubuntu@hadoop-coc-2:~/Programs/hadoop$ hdfs dfs -ls /
> Java HotSpot(TM) Client VM warning: You have loaded library /home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
> It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
> 15/03/03 05:09:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
>
>
>
> --
> --
>
>
Re: Can't list files in a federation of HDFS
Posted by Azuryy Yu <az...@gmail.com>.
For HDFS federation, data share all datanodes, but namespace is separate,
so did you write some data on hadoop-coc-2 namespace? you don't need to
login hadoop-coc-2 then write data, just config a new client, which connect
to hadoop-coc-2 for write.
On Tue, Mar 3, 2015 at 6:20 PM, xeonmailinglist <xe...@gmail.com>
wrote:
> Hi,
>
> I have configured in 2 hosts (hadooop-coc-1, and hadoop-coc-2) a
> federation of HDFS. In the configuration, I have set a namespace in each
> host, and a single data node (see image). The service is running properly.
> You can check the output of the jps commands in [1].
>
> The strange part is that, when I list the files, they do not appear in
> hadoop-coc-2. You can check the output in [2]. Why this happens?
>
> [image: hdfsfederation]
>
> [1]: jps output
>
> xubuntu@hadoop-coc-1:~/Programs/hadoop$ jps
> 21538 NameNode
> 21773 DataNode
>
> xubuntu@hadoop-coc-2:~/Programs/hadoop$ jps
> 2342 NameNode
>
> [2]: hdfs dfs -ls / output
>
> xubuntu@hadoop-coc-1:~/Programs/hadoop$ hdfs dfs -ls /
> Java HotSpot(TM) Client VM warning: You have loaded library /home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
> It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
> 15/03/03 05:09:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> Found 1 items
> drwxr-xr-x - xubuntu supergroup 0 2015-03-03 04:47 /input1
>
> xubuntu@hadoop-coc-2:~/Programs/hadoop$ hdfs dfs -ls /
> Java HotSpot(TM) Client VM warning: You have loaded library /home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
> It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
> 15/03/03 05:09:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
>
>
>
> --
> --
>
>
Re: Can't list files in a federation of HDFS
Posted by Azuryy Yu <az...@gmail.com>.
For HDFS federation, data share all datanodes, but namespace is separate,
so did you write some data on hadoop-coc-2 namespace? you don't need to
login hadoop-coc-2 then write data, just config a new client, which connect
to hadoop-coc-2 for write.
On Tue, Mar 3, 2015 at 6:20 PM, xeonmailinglist <xe...@gmail.com>
wrote:
> Hi,
>
> I have configured in 2 hosts (hadooop-coc-1, and hadoop-coc-2) a
> federation of HDFS. In the configuration, I have set a namespace in each
> host, and a single data node (see image). The service is running properly.
> You can check the output of the jps commands in [1].
>
> The strange part is that, when I list the files, they do not appear in
> hadoop-coc-2. You can check the output in [2]. Why this happens?
>
> [image: hdfsfederation]
>
> [1]: jps output
>
> xubuntu@hadoop-coc-1:~/Programs/hadoop$ jps
> 21538 NameNode
> 21773 DataNode
>
> xubuntu@hadoop-coc-2:~/Programs/hadoop$ jps
> 2342 NameNode
>
> [2]: hdfs dfs -ls / output
>
> xubuntu@hadoop-coc-1:~/Programs/hadoop$ hdfs dfs -ls /
> Java HotSpot(TM) Client VM warning: You have loaded library /home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
> It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
> 15/03/03 05:09:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> Found 1 items
> drwxr-xr-x - xubuntu supergroup 0 2015-03-03 04:47 /input1
>
> xubuntu@hadoop-coc-2:~/Programs/hadoop$ hdfs dfs -ls /
> Java HotSpot(TM) Client VM warning: You have loaded library /home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
> It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
> 15/03/03 05:09:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
>
>
>
> --
> --
>
>
Re: Can't list files in a federation of HDFS
Posted by Azuryy Yu <az...@gmail.com>.
For HDFS federation, data share all datanodes, but namespace is separate,
so did you write some data on hadoop-coc-2 namespace? you don't need to
login hadoop-coc-2 then write data, just config a new client, which connect
to hadoop-coc-2 for write.
On Tue, Mar 3, 2015 at 6:20 PM, xeonmailinglist <xe...@gmail.com>
wrote:
> Hi,
>
> I have configured in 2 hosts (hadooop-coc-1, and hadoop-coc-2) a
> federation of HDFS. In the configuration, I have set a namespace in each
> host, and a single data node (see image). The service is running properly.
> You can check the output of the jps commands in [1].
>
> The strange part is that, when I list the files, they do not appear in
> hadoop-coc-2. You can check the output in [2]. Why this happens?
>
> [image: hdfsfederation]
>
> [1]: jps output
>
> xubuntu@hadoop-coc-1:~/Programs/hadoop$ jps
> 21538 NameNode
> 21773 DataNode
>
> xubuntu@hadoop-coc-2:~/Programs/hadoop$ jps
> 2342 NameNode
>
> [2]: hdfs dfs -ls / output
>
> xubuntu@hadoop-coc-1:~/Programs/hadoop$ hdfs dfs -ls /
> Java HotSpot(TM) Client VM warning: You have loaded library /home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
> It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
> 15/03/03 05:09:04 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> Found 1 items
> drwxr-xr-x - xubuntu supergroup 0 2015-03-03 04:47 /input1
>
> xubuntu@hadoop-coc-2:~/Programs/hadoop$ hdfs dfs -ls /
> Java HotSpot(TM) Client VM warning: You have loaded library /home/xubuntu/Programs/hadoop-2.6.0/lib/native/libhadoop.so which might have disabled stack guard. The VM will try to fix the stack guard now.
> It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
> 15/03/03 05:09:07 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
>
>
>
> --
> --
>
>