You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@tez.apache.org by Chen He <ai...@gmail.com> on 2014/06/07 00:20:56 UTC

why are two Fetcher classes in Tez

one in in org.apache.tez.runtime.library.common.shuffle.impl
the other is in org.apache.tez.runtime.library.shuffle.common

Why don't we merge them together?

Looks like the second one is used in broadcastShuffleManager
but the first one is used in other shuffleManager.

There should be a way to merge them together, reuse some code,
 and avoid confusion.

Regards!

Chen

Re: why are two Fetcher classes in Tez

Posted by Siddharth Seth <ss...@apache.org>.

Chen, There's a jira open somewhere to merge the Fetchers and Shuffle
managers. It does need to be done at some point.

On Fri, Jun 6, 2014 at 3:20 PM, Chen He <ai...@gmail.com> wrote:

> one in in org.apache.tez.runtime.library.common.shuffle.impl
> the other is in org.apache.tez.runtime.library.shuffle.common
>
> Why don't we merge them together?
>
> Looks like the second one is used in broadcastShuffleManager
> but the first one is used in other shuffleManager.
>
> There should be a way to merge them together, reuse some code,
>  and avoid confusion.
>
> Regards!
>
> Chen
>