You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@pig.apache.org by "Cheolsoo Park (JIRA)" <ji...@apache.org> on 2014/01/06 02:28:50 UTC

[jira] [Created] (PIG-3648) Make the sample size for RandomSampleLoader configurable

Cheolsoo Park created PIG-3648:
----------------------------------

             Summary: Make the sample size for RandomSampleLoader configurable
                 Key: PIG-3648
                 URL: https://issues.apache.org/jira/browse/PIG-3648
             Project: Pig
          Issue Type: Improvement
          Components: impl
            Reporter: Cheolsoo Park
            Assignee: Cheolsoo Park
            Priority: Minor
             Fix For: 0.13.0


Pig uses RandomSampleLoader for range partitioning in order-by. But since the sample size is hardcoded as 100, volatility in the variance of the results  increases when sorting a large number of rows (e.g. 10M+).

It would be nice if the sample size could be configurable via Pig properties.





--
This message was sent by Atlassian JIRA
(v6.1.5#6160)