You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Steven Phillips <sp...@maprtech.com> on 2015/04/10 00:55:05 UTC
Review Request 33001: DRILL-2725: Faster work assignment logic
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33001/
-----------------------------------------------------------
Review request for drill, Jacques Nadeau and Jason Altekruse.
Repository: drill-git
Description
-------
The current AssignmentCreator logic for assigning work to drillbits takes a non-negligible amount of time once the number of work units is more than a few thousand
We need a new algorithm to that will cut this time down to less than a second, even for tables with more than 100K files.
Diffs
-----
exec/java-exec/src/main/java/org/apache/drill/exec/store/schedule/AssignmentCreator.java 7e9c4c9
Diff: https://reviews.apache.org/r/33001/diff/
Testing
-------
Thanks,
Steven Phillips
Re: Review Request 33001: DRILL-2725: Faster work assignment logic
Posted by Jason Altekruse <al...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33001/#review80149
-----------------------------------------------------------
Ship it!
exec/java-exec/src/main/java/org/apache/drill/exec/store/schedule/AssignmentCreator.java
<https://reviews.apache.org/r/33001/#comment129919>
Is there any particular reason for using the address as a secondary comparison. Is this supposed to help break ties to prevent skew if we have a lot of similar localities on different nodes?
exec/java-exec/src/main/java/org/apache/drill/exec/store/schedule/AssignmentCreator.java
<https://reviews.apache.org/r/33001/#comment129922>
spelling - corresponding
- Jason Altekruse
On April 9, 2015, 10:55 p.m., Steven Phillips wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33001/
> -----------------------------------------------------------
>
> (Updated April 9, 2015, 10:55 p.m.)
>
>
> Review request for drill, Jacques Nadeau and Jason Altekruse.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> The current AssignmentCreator logic for assigning work to drillbits takes a non-negligible amount of time once the number of work units is more than a few thousand
> We need a new algorithm to that will cut this time down to less than a second, even for tables with more than 100K files.
>
>
> Diffs
> -----
>
> exec/java-exec/src/main/java/org/apache/drill/exec/store/schedule/AssignmentCreator.java 7e9c4c9
>
> Diff: https://reviews.apache.org/r/33001/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Steven Phillips
>
>