You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Stefan Groschupf <sg...@media-style.com> on 2006/03/27 01:48:55 UTC
how blocks are compared
Hi,
I do not clearly understand how the dfs compare blocks.
The blockId is randomly generated but the compare method looks like:
if (getBlockId() < b.getBlockId()) {
return -1;
} else if (getBlockId() == b.getBlockId()) {
return 0;
} else {
return 1;
}
That wouldn't be a big problem if this would be used in the
FSNamesystem#processBlock
if (cmp == 0) {
// Do nothing, blocks are the same
...
} else if (cmp < 0) {
// The old report has a block the new one does not
removeStoredBlock(oldReport[oldPos], node);
...
} else {
// The new report has a block the old one does not
addStoredBlock(newReport[newPos], node);
I currently just miss some glue how result of a compare of two random
numbers can be used to decide if a block can be removed or not.
Any hints what I'm missing?
Thanks.
Stefan
---------------------------------------------
blog: http://www.find23.org
company: http://www.media-style.com
Re: how blocks are compared
Posted by Stefan Groschupf <sg...@media-style.com>.
Hi Andrzej,
I see!
Thanks for the hint.
Stefan
Am 27.03.2006 um 12:24 schrieb Andrzej Bialecki:
> Stefan Groschupf wrote:
>> Hi,
>> I do not clearly understand how the dfs compare blocks.
>
>> I currently just miss some glue how result of a compare of two
>> random numbers can be used to decide if a block can be removed or
>> not.
>>
>> Any hints what I'm missing?
>
> Block reports are sorted according to the same compateTo (in
> TreeSet.toArray()). It's true that IDs are random, but if both
> oldReport and newReport lists are sorted, like this:
>
> OLD NEW
> 1 2
> 2 3
> 3 4
> 5 5
>
> Then the sequence of compares and pointer advances looks like this
>
> old[oldPos=0].compareTo(new[newPos=0]) == -1; remove(old[0]); oldPos
> ++;
> old[oldPos=1].compareTo(new[newPos=0]) == 0; /* nothing */; oldPos+
> +; newPos++;
> old[oldPos=2].compareTo(new[newPos=1]) == 0; /* nothing */; oldPos+
> +; newPos++;
> old[oldPos=3].compareTo(new[newPos=2]) == 1; add(new[2]); newPos++;
> old[oldPos=3].compareTo(new[newPos=3]) == 0; /* nothing */; oldPos+
> +; newPos++;
>
> --
> Best regards,
> Andrzej Bialecki <><
> ___. ___ ___ ___ _ _ __________________________________
> [__ || __|__/|__||\/| Information Retrieval, Semantic Web
> ___|||__|| \| || | Embedded Unix, System Integration
> http://www.sigram.com Contact: info at sigram dot com
>
>
>
---------------------------------------------------------------
company: http://www.media-style.com
forum: http://www.text-mining.org
blog: http://www.find23.net
Re: how blocks are compared
Posted by Andrzej Bialecki <ab...@getopt.org>.
Stefan Groschupf wrote:
> Hi,
> I do not clearly understand how the dfs compare blocks.
> I currently just miss some glue how result of a compare of two random
> numbers can be used to decide if a block can be removed or not.
>
> Any hints what I'm missing?
Block reports are sorted according to the same compateTo (in
TreeSet.toArray()). It's true that IDs are random, but if both oldReport
and newReport lists are sorted, like this:
OLD NEW
1 2
2 3
3 4
5 5
Then the sequence of compares and pointer advances looks like this
old[oldPos=0].compareTo(new[newPos=0]) == -1; remove(old[0]); oldPos++;
old[oldPos=1].compareTo(new[newPos=0]) == 0; /* nothing */; oldPos++;
newPos++;
old[oldPos=2].compareTo(new[newPos=1]) == 0; /* nothing */; oldPos++;
newPos++;
old[oldPos=3].compareTo(new[newPos=2]) == 1; add(new[2]); newPos++;
old[oldPos=3].compareTo(new[newPos=3]) == 0; /* nothing */; oldPos++;
newPos++;
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com