You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Selwa <el...@gmail.com> on 2012/02/12 02:41:08 UTC

datanode to blocks mapping

Hello there,
currently i am working on research about the block placement policy of HDFS.
I want to find the blocks mapping of the pseudo cluster.
then i request the name node for this data structure by the following code

       BlocksMap blocksMap = new BlocksMap(16, 0.75f);
      Collection<BlockInfo>blocks = blocksMap.getBlocks();
              but the collection does not hold any thing - always it is
empty
             blocks.isempty() is becoming true even i saved some files on
the pseudo -cluster.
what is wrong with this code?
is there any other way to find block mapping?

thanks

Selwa

--
View this message in context: http://lucene.472066.n3.nabble.com/datanode-to-blocks-mapping-tp3736421p3736421.html
Sent from the Hadoop lucene-dev mailing list archive at Nabble.com.

Re: datanode to blocks mapping

Posted by Ravi Prakash <ra...@gmail.com>.
Selwa,

I'm glad I could help. I suggest you import the source code in an IDE (we
use Eclipse) and navigate around it. The outline view in Eclipse is a
lifesaver :)

BlocksMap does not seem to have that API. I would try FSNamesystem on the
namenode. On the client side, the DFSClient class is what opens a file (and
takes care of all the block-mapping).

Hope this helps,
Ravi.



On Wed, Feb 15, 2012 at 6:56 PM, Selwa <el...@gmail.com> wrote:

> Ravi,
>
> thank u very much for your reply.
> i am using version 0.21.0 . hope your comment will help me a lot.
> BTW, after i got the blocksMap, can i request this blocksMap to return
> blocks of a given file.
> i mean how to map a filename to its blocks?
>
> Best wishes,
> Selwa
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/datanode-to-blocks-mapping-tp3736421p3748957.html
> Sent from the Hadoop lucene-dev mailing list archive at Nabble.com.
>

Re: datanode to blocks mapping

Posted by Selwa <el...@gmail.com>.
Ravi,

thank u very much for your reply.
i am using version 0.21.0 . hope your comment will help me a lot.
BTW, after i got the blocksMap, can i request this blocksMap to return
blocks of a given file.
i mean how to map a filename to its blocks?

Best wishes,
Selwa

--
View this message in context: http://lucene.472066.n3.nabble.com/datanode-to-blocks-mapping-tp3736421p3748957.html
Sent from the Hadoop lucene-dev mailing list archive at Nabble.com.

Re: datanode to blocks mapping

Posted by Ravi Prakash <ra...@gmail.com>.
Selwa,

You are *creating* a new BlocksMap with the first line. It is NOT an API to
get the blockmap from the namenode. The Namenode stores its blocks in a
Blockmap in the BlockManager (for 0.23) or FSNameSystem (for 0.20)

Which version are you using?

Regards,
Ravi.

On Sat, Feb 11, 2012 at 7:41 PM, Selwa <el...@gmail.com> wrote:

> Hello there,
> currently i am working on research about the block placement policy of
> HDFS.
> I want to find the blocks mapping of the pseudo cluster.
> then i request the name node for this data structure by the following code
>
>       BlocksMap blocksMap = new BlocksMap(16, 0.75f);
>      Collection<BlockInfo>blocks = blocksMap.getBlocks();
>              but the collection does not hold any thing - always it is
> empty
>             blocks.isempty() is becoming true even i saved some files on
> the pseudo -cluster.
> what is wrong with this code?
> is there any other way to find block mapping?
>
> thanks
>
> Selwa
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/datanode-to-blocks-mapping-tp3736421p3736421.html
> Sent from the Hadoop lucene-dev mailing list archive at Nabble.com.
>