Re: MapRed Generator

[Doug Cutting <cu...@nutch.org>]

Are you crawling only a single host?  If so, I can see how this would 
happen.  Using two hosts to crawl a single host is probably not a good 
idea anyway, no?

Doug

Anton Potehin wrote:
> Class Generator
> 
>  
> 
> We have 2 Reduce Tasks
> 
>  
> 
> Limit = TopN / 2;
> 
>  
> 
> Generator.Selector.Reduce for first task receive all K,V pairs from
> maps, but select only half of them (work limit)
> 
> Generator.Selector.Reduce for second task doesn't receive pairs at all!
> 
>  
> 
> In result on output we have half of mapped pairs!
> 
>  
> 
> This is bug? Or I'm don't understand something?
> 
>