You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Daniel Dai (JIRA)" <ji...@apache.org> on 2010/01/16 03:12:54 UTC
[jira] Created: (PIG-1195) InternalSortedBag should take care of
sort order
InternalSortedBag should take care of sort order
------------------------------------------------
Key: PIG-1195
URL: https://issues.apache.org/jira/browse/PIG-1195
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.6.0
Reporter: Daniel Dai
Fix For: 0.6.0
InternalSortedBag always use ascending order. We shall obey the sort order as specified in the script.
For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on InternalSortedBag to sort):
{code}
A = load 'input' as (a0:int);
B = group A ALL;
C = foreach B {
D = order A by a0 desc;
generate D;
};
dump C;
{code}
If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1195) InternalSortedBag should take care of
sort order
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801342#action_12801342 ]
Hadoop QA commented on PIG-1195:
--------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12430532/PIG-1195-2.patch
against trunk revision 899502.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/182/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/182/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/182/console
This message is automatically generated.
> InternalSortedBag should take care of sort order
> ------------------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch
>
>
> InternalSortedBag always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on InternalSortedBag to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1195) InternalSortedBag should take care of
sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1195:
----------------------------
Attachment: PIG-1195-1.patch
> InternalSortedBag should take care of sort order
> ------------------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch
>
>
> InternalSortedBag always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on InternalSortedBag to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1195) InternalSortedBag should take care of
sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1195:
----------------------------
Status: Patch Available (was: Open)
> InternalSortedBag should take care of sort order
> ------------------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch
>
>
> InternalSortedBag always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on InternalSortedBag to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1195) POSort should take care of sort order
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802653#action_12802653 ]
Hadoop QA commented on PIG-1195:
--------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12430787/PIG-1195-3.patch
against trunk revision 900926.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/184/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/184/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/184/console
This message is automatically generated.
> POSort should take care of sort order
> -------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch, PIG-1195-3.patch, PIG-1195-4.patch
>
>
> POSort always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1195) POSort should take care of sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1195:
----------------------------
Resolution: Fixed
Release Note: Patch committed to both trunk and 0.6 branch.
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
> POSort should take care of sort order
> -------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch, PIG-1195-3.patch, PIG-1195-4.patch
>
>
> POSort always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1195) POSort should take care of sort order
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802569#action_12802569 ]
Pradeep Kamath commented on PIG-1195:
-------------------------------------
+1 for commit
> POSort should take care of sort order
> -------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch, PIG-1195-3.patch, PIG-1195-4.patch
>
>
> POSort always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1195) POSort should take care of sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1195:
----------------------------
Status: Patch Available (was: Open)
> POSort should take care of sort order
> -------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch, PIG-1195-3.patch
>
>
> POSort always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1195) InternalSortedBag should take care of
sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1195:
----------------------------
Status: Open (was: Patch Available)
> InternalSortedBag should take care of sort order
> ------------------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch
>
>
> InternalSortedBag always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on InternalSortedBag to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1195) POSort should take care of sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1195:
----------------------------
Status: Open (was: Patch Available)
> POSort should take care of sort order
> -------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch, PIG-1195-3.patch
>
>
> POSort always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1195) InternalSortedBag should take care of
sort order
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802397#action_12802397 ]
Alan Gates commented on PIG-1195:
---------------------------------
The sorting algorithm in DefaultComparator does not match the sorting algorithm in DefaultTuple.compare.
The algorithm used here first compares the values of each column, and only considers the overall size of the tuples once one tuple has run out of fields. The algorithm used in DefaultTuple.compare first compares tuple size, then individual column values. So in this algorithm (5, 3) > (4, 3, 1), but in DefaultTuple's algorithm (5, 3) < (4, 3, 1). We should use the same algorithm in both places.
> InternalSortedBag should take care of sort order
> ------------------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch
>
>
> InternalSortedBag always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on InternalSortedBag to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1195) POSort should take care of sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1195:
----------------------------
Attachment: PIG-1195-3.patch
> POSort should take care of sort order
> -------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch, PIG-1195-3.patch
>
>
> POSort always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1195) InternalSortedBag should take care of
sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1195:
----------------------------
Attachment: PIG-1195-2.patch
Change the comparison logic a little bit.
> InternalSortedBag should take care of sort order
> ------------------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch
>
>
> InternalSortedBag always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on InternalSortedBag to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1195) POSort should take care of sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1195:
----------------------------
Description:
POSort always use ascending order. We shall obey the sort order as specified in the script.
For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
{code}
A = load 'input' as (a0:int);
B = group A ALL;
C = foreach B {
D = order A by a0 desc;
generate D;
};
dump C;
{code}
If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
The sort order for D is ascending.
was:
InternalSortedBag always use ascending order. We shall obey the sort order as specified in the script.
For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on InternalSortedBag to sort):
{code}
A = load 'input' as (a0:int);
B = group A ALL;
C = foreach B {
D = order A by a0 desc;
generate D;
};
dump C;
{code}
If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
The sort order for D is ascending.
Summary: POSort should take care of sort order (was: InternalSortedBag should take care of sort order)
> POSort should take care of sort order
> -------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch
>
>
> POSort always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1195) POSort should take care of sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1195:
----------------------------
Attachment: PIG-1195-4.patch
Use createInputFile instead of generateURI in test as per Pradeep's comment.
> POSort should take care of sort order
> -------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch, PIG-1195-3.patch, PIG-1195-4.patch
>
>
> POSort always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1195) POSort should take care of sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai reassigned PIG-1195:
-------------------------------
Assignee: Daniel Dai
> POSort should take care of sort order
> -------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch, PIG-1195-3.patch
>
>
> POSort always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1195) POSort should take care of sort order
Posted by "Ying He (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802439#action_12802439 ]
Ying He commented on PIG-1195:
------------------------------
+1
> POSort should take care of sort order
> -------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch, PIG-1195-3.patch
>
>
> POSort always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1195) POSort should take care of sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802421#action_12802421 ]
Daniel Dai commented on PIG-1195:
---------------------------------
Thanks Alan. Actually after talking with Ying, I realize originally POSort use mComparator to do the sorting. This code is broken in the current code, we should fix this rather than introduce something new.
> POSort should take care of sort order
> -------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch
>
>
> POSort always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1195) InternalSortedBag should take care of
sort order
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801162#action_12801162 ]
Hadoop QA commented on PIG-1195:
--------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12430478/PIG-1195-1.patch
against trunk revision 899502.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/179/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/179/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h8.grid.sp2.yahoo.net/179/console
This message is automatically generated.
> InternalSortedBag should take care of sort order
> ------------------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch
>
>
> InternalSortedBag always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on InternalSortedBag to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1195) POSort should take care of sort order
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12802552#action_12802552 ]
Pradeep Kamath commented on PIG-1195:
-------------------------------------
One comment:
- In unit test I would recommend you use Util.createInputFile() method which take the minicluster as an input arg and creates a input file on the cluster. Also delete the file in a finally using Util.deleteFile() - the Util.generateURI() is something which will create problems while merging to load-store-redesign branch.
Otherwise +1
> POSort should take care of sort order
> -------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch, PIG-1195-2.patch, PIG-1195-3.patch
>
>
> POSort always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on POSort to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1195) InternalSortedBag should take care of
sort order
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1195:
----------------------------
Status: Patch Available (was: Open)
> InternalSortedBag should take care of sort order
> ------------------------------------------------
>
> Key: PIG-1195
> URL: https://issues.apache.org/jira/browse/PIG-1195
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.6.0
> Reporter: Daniel Dai
> Fix For: 0.6.0
>
> Attachments: PIG-1195-1.patch
>
>
> InternalSortedBag always use ascending order. We shall obey the sort order as specified in the script.
> For example, the following script does not do the right thing if we turn off secondary sort (which means, we will rely on InternalSortedBag to sort):
> {code}
> A = load 'input' as (a0:int);
> B = group A ALL;
> C = foreach B {
> D = order A by a0 desc;
> generate D;
> };
> dump C;
> {code}
> If we run it using the command line "java -Xmx512m -Dpig.exec.nosecondarykey=true -jar pig.jar 1.pig".
> The sort order for D is ascending.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.