You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Sriranjan Manjunath (JIRA)" <ji...@apache.org> on 2009/07/17 02:25:14 UTC

[jira] Created: (PIG-890) Create a sampler interface and improve the skewed join sampler

Create a sampler interface and improve the skewed join sampler
--------------------------------------------------------------

                 Key: PIG-890
                 URL: https://issues.apache.org/jira/browse/PIG-890
             Project: Pig
          Issue Type: Improvement
            Reporter: Sriranjan Manjunath


We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Sriranjan Manjunath (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sriranjan Manjunath updated PIG-890:
------------------------------------

    Attachment: sampler.patch

The attached file has the redesigned sampler interface. Skewed join now uses a trivial implementation of the poisson sampling mechanism.

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Sriranjan Manjunath (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sriranjan Manjunath updated PIG-890:
------------------------------------

    Status: Patch Available  (was: Open)

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: samplerinterface.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Sriranjan Manjunath (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sriranjan Manjunath updated PIG-890:
------------------------------------

    Attachment: samplerinterface.patch

Fixed the review comments

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: samplerinterface.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Sriranjan Manjunath (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sriranjan Manjunath updated PIG-890:
------------------------------------

    Attachment:     (was: sampler.patch)

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Sriranjan Manjunath (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742136#action_12742136 ] 

Sriranjan Manjunath commented on PIG-890:
-----------------------------------------

Let me know if you think that this requires a test case and I will be happy to include it.

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745843#action_12745843 ] 

Daniel Dai commented on PIG-890:
--------------------------------

In your wiki, "For an 1TB file running on nodes which have 512 MB of memory, assuming a conversion factor of 2, the number of base samples turn out to be 4000", can you give more explanation on that?

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai updated PIG-890:
---------------------------

       Resolution: Fixed
    Fix Version/s: 0.4.0
           Status: Resolved  (was: Patch Available)

Patch committed. Unit test failure is not related to this patch. Thanks Sri. 

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>             Fix For: 0.4.0
>
>         Attachments: samplerinterface.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745830#action_12745830 ] 

Daniel Dai commented on PIG-890:
--------------------------------

Comments:
1. Can you include unit test?
2. PoissonSampleLoader.java
{noformat}
		try {
			numSplits = Integer.valueOf(pcProps.getProperty(MAPSPLITS_COUNT));
		} catch (NumberFormatException e) {
			numSplits = 1;
		}
{noformat}
We shall throw exception rather than continue.
Same to 
{noformat}
		try {
			float f = (Runtime.getRuntime().maxMemory() * heapPerc) / (float) (FileLocalizer.getSize(fname) * convFactor);
			baseNumSamples = (long) Math.ceil(1.0 / f);
		} catch (IOException e) {
			baseNumSamples = 1; // default value 
		}
{noformat}
3. Are PoissonSampleLoader.next and PoissonSampleLoader.bindTo the same with RandomSampleLoader? If so, we shall put them in base class rather than copy
4. For DEFAULT_SAMPLE_RATE, can you provide some other values in the comment, such as confidence 90%, 85%, and also put a link of how to get these magic numbers. I know this is Poisson cdf, but it is better to have something we can check really quick

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750701#action_12750701 ] 

Daniel Dai commented on PIG-890:
--------------------------------

+1 for the patch.

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: samplerinterface.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742118#action_12742118 ] 

Hadoop QA commented on PIG-890:
-------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12416250/sampler.patch
  against trunk revision 801865.

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no tests are needed for this patch.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    -1 findbugs.  The patch appears to introduce 6 new Findbugs warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/157/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/157/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/157/console

This message is automatically generated.

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Sriranjan Manjunath (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sriranjan Manjunath updated PIG-890:
------------------------------------

    Status: Open  (was: Patch Available)

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12745840#action_12745840 ] 

Daniel Dai commented on PIG-890:
--------------------------------

Forget 4, I see it in [PigSampler|http://wiki.apache.org/pig/PigSampler]

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750715#action_12750715 ] 

Hadoop QA commented on PIG-890:
-------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12418432/samplerinterface.patch
  against trunk revision 810677.

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 4 new or modified tests.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    -1 core tests.  The patch failed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/10/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/10/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/10/console

This message is automatically generated.

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: samplerinterface.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Sriranjan Manjunath (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sriranjan Manjunath updated PIG-890:
------------------------------------

    Status: Open  (was: Patch Available)

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Assigned: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alan Gates reassigned PIG-890:
------------------------------

    Assignee: Sriranjan Manjunath

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>            Assignee: Sriranjan Manjunath
>             Fix For: 0.4.0
>
>         Attachments: samplerinterface.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742203#action_12742203 ] 

Hadoop QA commented on PIG-890:
-------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12416267/sampler.patch
  against trunk revision 803312.

    +1 @author.  The patch does not contain any @author tags.

    -1 tests included.  The patch doesn't appear to include any new or modified tests.
                        Please justify why no tests are needed for this patch.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 findbugs.  The patch does not introduce any new Findbugs warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit warnings.

    +1 core tests.  The patch passed core unit tests.

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/159/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/159/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/159/console

This message is automatically generated.

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Sriranjan Manjunath (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sriranjan Manjunath updated PIG-890:
------------------------------------

    Status: Patch Available  (was: Open)

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Sriranjan Manjunath (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sriranjan Manjunath updated PIG-890:
------------------------------------

    Status: Patch Available  (was: Open)

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Sriranjan Manjunath (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sriranjan Manjunath updated PIG-890:
------------------------------------

    Attachment: sampler.patch

Made some constants static to clear the findbugs warnings. This patch does not warrant a new test case since it only affects the performance of the skewed join sampler and SkewedJoin test case already handles the correctness of the join.

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: sampler.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (PIG-890) Create a sampler interface and improve the skewed join sampler

Posted by "Sriranjan Manjunath (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/PIG-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sriranjan Manjunath updated PIG-890:
------------------------------------

    Attachment:     (was: sampler.patch)

> Create a sampler interface and improve the skewed join sampler
> --------------------------------------------------------------
>
>                 Key: PIG-890
>                 URL: https://issues.apache.org/jira/browse/PIG-890
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Sriranjan Manjunath
>         Attachments: samplerinterface.patch
>
>
> We need a different sampler for order by and skewed join. We thus need a better sampling interface. The design of the same is described here: http://wiki.apache.org/pig/PigSampler

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.