You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by lulynn_2008 <lu...@163.com> on 2012/08/06 15:57:07 UTC
[pig-0.10.0 e2e test]: some tests failed with "Sort check failed"
error
Hi All,
I am running pig-0.10.0 e2e test with pig-0.9.2 and hadoop-1.0.3. There are 12 tests failed with "Sort check failed" error.
I list Order_6 as a example:
Error info:
Going to run sort check command: sort -cs -t +0 -1 ./out/pigtest/pig/pig.1344259341/Order_6.out/out_original
/bin/sort: ./out/pigtest/pig/pig.1344259341/Order_6.out/out_original:2: disorder: alice allen 36 2.27
Sort check failed
Related files:
studenttab10k file:
ulysses thompson 64 1.90
katie carson 25 3.65
luke king 65 0.73
holly davidson 57 2.43
fred miller 55 3.77
holly white 43 0.24
luke steinbeck 51 1.14
nick underhill 31 2.46
holly davidson 59 1.26
calvin brown 56 0.72
rachel robinson 62 2.25
tom carson 35 0.56
tom johnson 72 0.99
irene garcia 54 1.06
oscar nixon 39 3.60
holly allen 32 2.58
oscar hernandez 19 0.05
alice ichabod 65 2.25
wendy thompson 30 2.39
priscilla hernandez 73 0.23
gabriella van buren 68 1.32
yuri thompson 42 3.65
yuri laertes 60 1.16
sarah young 23 2.76
zach white 32 0.20
nick van buren 68 1.75
xavier underhill 41 1.51
bob ichabod 56 2.81
zach steinbeck 61 2.22
alice garcia 42 2.03
jessica king 29 3.61
calvin nixon 37 0.30
fred polk 66 3.69
bob zipper 40 0.28
alice young 75 0.31
nick underhill 37 1.65
mike white 57 0.69
calvin ovid 41 3.02
fred steinbeck 47 3.57
sarah ovid 65 0.00
wendy nixon 63 0.6
Order_6.pig file:
a = load '/user/pig/tests/data/singlefile/studenttab10k';
c = order a by $0;
store c into '/user/pig/out/biadmin.1344259341/Order_6.out';
out.original file;
alice allen 74 0.02
alice allen 36 2.27
alice allen 20 2.47
alice allen 72 1.75
alice allen 42 2.46
alice allen 43 0.88
alice allen 27 1.95
alice allen 74 2.40
alice allen 72 3.63
alice allen 46 3.97
alice allen 77 2.52
alice allen 68 3.51
alice allen 34 1.23
alice allen 51 1.08
alice allen 68 3.39
alice allen 38 1.80
alice allen 38 0.81
alice allen 45 2.80
alice allen 19 1.93
alice brown 59 2.94
alice brown 66 1.57
alice brown 73 1.75
alice brown 69 3.04
...
out_sorted:
alice allen 19 1.93
alice allen 20 2.47
alice allen 27 1.95
alice allen 34 1.23
alice allen 36 2.27
alice allen 38 0.81
alice allen 38 1.80
alice allen 42 2.46
alice allen 43 0.88
alice allen 45 2.80
alice allen 46 3.97
alice allen 51 1.08
alice allen 68 3.39
alice allen 68 3.51
alice allen 72 1.75
alice allen 72 3.63
alice allen 74 0.02
alice allen 74 2.40
alice allen 77 2.52
alice brown 18 0.68
alice brown 21 2.57
alice brown 27 3.89
alice brown 38 3.65
alice brown 44 3.82
alice brown 59 2.94
alice brown 61 2.02
alice brown 66 1.57
alice brown 69 3.04
alice brown 70 1.27
alice brown 73 1.75
alice brown 73 3.36
alice carson 19 1.88
alice carson 22 2.82
...
Re:Re: [pig-0.10.0 e2e test]: some tests failed with "Sort check
failed" error
Posted by lulynn_2008 <lu...@163.com>.
The patch in PIG-2498 works. Thanks.
At 2012-08-07 01:05:15,"Cheolsoo Park" <ch...@cloudera.com> wrote:
>Hi,
>
>I believe that you're hitting the same problem as PIG-2498:
>https://issues.apache.org/jira/browse/PIG-2498
>
>Please see if that patch fixes your problem.
>
>Or I believe that this should also be fixed by PIG-2782 where old sort
>notation is replaced with new one:
>https://issues.apache.org/jira/browse/PIG-2782
>
>Thanks!
>Cheolsoo
>
>On Mon, Aug 6, 2012 at 6:57 AM, lulynn_2008 <lu...@163.com> wrote:
>
>> Hi All,
>> I am running pig-0.10.0 e2e test with pig-0.9.2 and hadoop-1.0.3. There
>> are 12 tests failed with "Sort check failed" error.
>>
>> I list Order_6 as a example:
>> Error info:
>> Going to run sort check command: sort -cs -t +0 -1
>> ./out/pigtest/pig/pig.1344259341/Order_6.out/out_original
>> /bin/sort: ./out/pigtest/pig/pig.1344259341/Order_6.out/out_original:2:
>> disorder: alice allen 36 2.27
>> Sort check failed
>>
>> Related files:
>> studenttab10k file:
>> ulysses thompson 64 1.90
>> katie carson 25 3.65
>> luke king 65 0.73
>> holly davidson 57 2.43
>> fred miller 55 3.77
>> holly white 43 0.24
>> luke steinbeck 51 1.14
>> nick underhill 31 2.46
>> holly davidson 59 1.26
>> calvin brown 56 0.72
>> rachel robinson 62 2.25
>> tom carson 35 0.56
>> tom johnson 72 0.99
>> irene garcia 54 1.06
>> oscar nixon 39 3.60
>> holly allen 32 2.58
>> oscar hernandez 19 0.05
>> alice ichabod 65 2.25
>> wendy thompson 30 2.39
>> priscilla hernandez 73 0.23
>> gabriella van buren 68 1.32
>> yuri thompson 42 3.65
>> yuri laertes 60 1.16
>> sarah young 23 2.76
>> zach white 32 0.20
>> nick van buren 68 1.75
>> xavier underhill 41 1.51
>> bob ichabod 56 2.81
>> zach steinbeck 61 2.22
>> alice garcia 42 2.03
>> jessica king 29 3.61
>> calvin nixon 37 0.30
>> fred polk 66 3.69
>> bob zipper 40 0.28
>> alice young 75 0.31
>> nick underhill 37 1.65
>> mike white 57 0.69
>> calvin ovid 41 3.02
>> fred steinbeck 47 3.57
>> sarah ovid 65 0.00
>> wendy nixon 63 0.6
>>
>> Order_6.pig file:
>> a = load '/user/pig/tests/data/singlefile/studenttab10k';
>> c = order a by $0;
>> store c into '/user/pig/out/biadmin.1344259341/Order_6.out';
>>
>> out.original file;
>> alice allen 74 0.02
>> alice allen 36 2.27
>> alice allen 20 2.47
>> alice allen 72 1.75
>> alice allen 42 2.46
>> alice allen 43 0.88
>> alice allen 27 1.95
>> alice allen 74 2.40
>> alice allen 72 3.63
>> alice allen 46 3.97
>> alice allen 77 2.52
>> alice allen 68 3.51
>> alice allen 34 1.23
>> alice allen 51 1.08
>> alice allen 68 3.39
>> alice allen 38 1.80
>> alice allen 38 0.81
>> alice allen 45 2.80
>> alice allen 19 1.93
>> alice brown 59 2.94
>> alice brown 66 1.57
>> alice brown 73 1.75
>> alice brown 69 3.04
>> ...
>>
>> out_sorted:
>> alice allen 19 1.93
>> alice allen 20 2.47
>> alice allen 27 1.95
>> alice allen 34 1.23
>> alice allen 36 2.27
>> alice allen 38 0.81
>> alice allen 38 1.80
>> alice allen 42 2.46
>> alice allen 43 0.88
>> alice allen 45 2.80
>> alice allen 46 3.97
>> alice allen 51 1.08
>> alice allen 68 3.39
>> alice allen 68 3.51
>> alice allen 72 1.75
>> alice allen 72 3.63
>> alice allen 74 0.02
>> alice allen 74 2.40
>> alice allen 77 2.52
>> alice brown 18 0.68
>> alice brown 21 2.57
>> alice brown 27 3.89
>> alice brown 38 3.65
>> alice brown 44 3.82
>> alice brown 59 2.94
>> alice brown 61 2.02
>> alice brown 66 1.57
>> alice brown 69 3.04
>> alice brown 70 1.27
>> alice brown 73 1.75
>> alice brown 73 3.36
>> alice carson 19 1.88
>> alice carson 22 2.82
>> ...
>>
>>
Re: [pig-0.10.0 e2e test]: some tests failed with "Sort check failed" error
Posted by Cheolsoo Park <ch...@cloudera.com>.
Hi,
I believe that you're hitting the same problem as PIG-2498:
https://issues.apache.org/jira/browse/PIG-2498
Please see if that patch fixes your problem.
Or I believe that this should also be fixed by PIG-2782 where old sort
notation is replaced with new one:
https://issues.apache.org/jira/browse/PIG-2782
Thanks!
Cheolsoo
On Mon, Aug 6, 2012 at 6:57 AM, lulynn_2008 <lu...@163.com> wrote:
> Hi All,
> I am running pig-0.10.0 e2e test with pig-0.9.2 and hadoop-1.0.3. There
> are 12 tests failed with "Sort check failed" error.
>
> I list Order_6 as a example:
> Error info:
> Going to run sort check command: sort -cs -t +0 -1
> ./out/pigtest/pig/pig.1344259341/Order_6.out/out_original
> /bin/sort: ./out/pigtest/pig/pig.1344259341/Order_6.out/out_original:2:
> disorder: alice allen 36 2.27
> Sort check failed
>
> Related files:
> studenttab10k file:
> ulysses thompson 64 1.90
> katie carson 25 3.65
> luke king 65 0.73
> holly davidson 57 2.43
> fred miller 55 3.77
> holly white 43 0.24
> luke steinbeck 51 1.14
> nick underhill 31 2.46
> holly davidson 59 1.26
> calvin brown 56 0.72
> rachel robinson 62 2.25
> tom carson 35 0.56
> tom johnson 72 0.99
> irene garcia 54 1.06
> oscar nixon 39 3.60
> holly allen 32 2.58
> oscar hernandez 19 0.05
> alice ichabod 65 2.25
> wendy thompson 30 2.39
> priscilla hernandez 73 0.23
> gabriella van buren 68 1.32
> yuri thompson 42 3.65
> yuri laertes 60 1.16
> sarah young 23 2.76
> zach white 32 0.20
> nick van buren 68 1.75
> xavier underhill 41 1.51
> bob ichabod 56 2.81
> zach steinbeck 61 2.22
> alice garcia 42 2.03
> jessica king 29 3.61
> calvin nixon 37 0.30
> fred polk 66 3.69
> bob zipper 40 0.28
> alice young 75 0.31
> nick underhill 37 1.65
> mike white 57 0.69
> calvin ovid 41 3.02
> fred steinbeck 47 3.57
> sarah ovid 65 0.00
> wendy nixon 63 0.6
>
> Order_6.pig file:
> a = load '/user/pig/tests/data/singlefile/studenttab10k';
> c = order a by $0;
> store c into '/user/pig/out/biadmin.1344259341/Order_6.out';
>
> out.original file;
> alice allen 74 0.02
> alice allen 36 2.27
> alice allen 20 2.47
> alice allen 72 1.75
> alice allen 42 2.46
> alice allen 43 0.88
> alice allen 27 1.95
> alice allen 74 2.40
> alice allen 72 3.63
> alice allen 46 3.97
> alice allen 77 2.52
> alice allen 68 3.51
> alice allen 34 1.23
> alice allen 51 1.08
> alice allen 68 3.39
> alice allen 38 1.80
> alice allen 38 0.81
> alice allen 45 2.80
> alice allen 19 1.93
> alice brown 59 2.94
> alice brown 66 1.57
> alice brown 73 1.75
> alice brown 69 3.04
> ...
>
> out_sorted:
> alice allen 19 1.93
> alice allen 20 2.47
> alice allen 27 1.95
> alice allen 34 1.23
> alice allen 36 2.27
> alice allen 38 0.81
> alice allen 38 1.80
> alice allen 42 2.46
> alice allen 43 0.88
> alice allen 45 2.80
> alice allen 46 3.97
> alice allen 51 1.08
> alice allen 68 3.39
> alice allen 68 3.51
> alice allen 72 1.75
> alice allen 72 3.63
> alice allen 74 0.02
> alice allen 74 2.40
> alice allen 77 2.52
> alice brown 18 0.68
> alice brown 21 2.57
> alice brown 27 3.89
> alice brown 38 3.65
> alice brown 44 3.82
> alice brown 59 2.94
> alice brown 61 2.02
> alice brown 66 1.57
> alice brown 69 3.04
> alice brown 70 1.27
> alice brown 73 1.75
> alice brown 73 3.36
> alice carson 19 1.88
> alice carson 22 2.82
> ...
>
>
Re:Re: [pig-0.10.0 e2e test]: some tests failed with "Sort check
failed" error
Posted by lulynn_2008 <lu...@163.com>.
pig-e2e test need an old version of pig. Please correct me if I missed anything. Thanks.
https://cwiki.apache.org/confluence/display/PIG/HowToTest
"Running the e2e tests requires three things: a cluster to run them on, an old version of Pig to use to generate expected results, and perl and a cpan module IPC::Run on your client machine."
At 2012-08-07 02:42:50,"Dmitriy Ryaboy" <dv...@gmail.com> wrote:
>I'm just curious, why do you expect pig 0.10 tests to succeed on 0.9.2?
>
>D
>
>On Mon, Aug 6, 2012 at 6:57 AM, lulynn_2008 <lu...@163.com> wrote:
>> Hi All,
>> I am running pig-0.10.0 e2e test with pig-0.9.2 and hadoop-1.0.3. There are 12 tests failed with "Sort check failed" error.
>>
>> I list Order_6 as a example:
>> Error info:
>> Going to run sort check command: sort -cs -t +0 -1 ./out/pigtest/pig/pig.1344259341/Order_6.out/out_original
>> /bin/sort: ./out/pigtest/pig/pig.1344259341/Order_6.out/out_original:2: disorder: alice allen 36 2.27
>> Sort check failed
>>
>> Related files:
>> studenttab10k file:
>> ulysses thompson 64 1.90
>> katie carson 25 3.65
>> luke king 65 0.73
>> holly davidson 57 2.43
>> fred miller 55 3.77
>> holly white 43 0.24
>> luke steinbeck 51 1.14
>> nick underhill 31 2.46
>> holly davidson 59 1.26
>> calvin brown 56 0.72
>> rachel robinson 62 2.25
>> tom carson 35 0.56
>> tom johnson 72 0.99
>> irene garcia 54 1.06
>> oscar nixon 39 3.60
>> holly allen 32 2.58
>> oscar hernandez 19 0.05
>> alice ichabod 65 2.25
>> wendy thompson 30 2.39
>> priscilla hernandez 73 0.23
>> gabriella van buren 68 1.32
>> yuri thompson 42 3.65
>> yuri laertes 60 1.16
>> sarah young 23 2.76
>> zach white 32 0.20
>> nick van buren 68 1.75
>> xavier underhill 41 1.51
>> bob ichabod 56 2.81
>> zach steinbeck 61 2.22
>> alice garcia 42 2.03
>> jessica king 29 3.61
>> calvin nixon 37 0.30
>> fred polk 66 3.69
>> bob zipper 40 0.28
>> alice young 75 0.31
>> nick underhill 37 1.65
>> mike white 57 0.69
>> calvin ovid 41 3.02
>> fred steinbeck 47 3.57
>> sarah ovid 65 0.00
>> wendy nixon 63 0.6
>>
>> Order_6.pig file:
>> a = load '/user/pig/tests/data/singlefile/studenttab10k';
>> c = order a by $0;
>> store c into '/user/pig/out/biadmin.1344259341/Order_6.out';
>>
>> out.original file;
>> alice allen 74 0.02
>> alice allen 36 2.27
>> alice allen 20 2.47
>> alice allen 72 1.75
>> alice allen 42 2.46
>> alice allen 43 0.88
>> alice allen 27 1.95
>> alice allen 74 2.40
>> alice allen 72 3.63
>> alice allen 46 3.97
>> alice allen 77 2.52
>> alice allen 68 3.51
>> alice allen 34 1.23
>> alice allen 51 1.08
>> alice allen 68 3.39
>> alice allen 38 1.80
>> alice allen 38 0.81
>> alice allen 45 2.80
>> alice allen 19 1.93
>> alice brown 59 2.94
>> alice brown 66 1.57
>> alice brown 73 1.75
>> alice brown 69 3.04
>> ...
>>
>> out_sorted:
>> alice allen 19 1.93
>> alice allen 20 2.47
>> alice allen 27 1.95
>> alice allen 34 1.23
>> alice allen 36 2.27
>> alice allen 38 0.81
>> alice allen 38 1.80
>> alice allen 42 2.46
>> alice allen 43 0.88
>> alice allen 45 2.80
>> alice allen 46 3.97
>> alice allen 51 1.08
>> alice allen 68 3.39
>> alice allen 68 3.51
>> alice allen 72 1.75
>> alice allen 72 3.63
>> alice allen 74 0.02
>> alice allen 74 2.40
>> alice allen 77 2.52
>> alice brown 18 0.68
>> alice brown 21 2.57
>> alice brown 27 3.89
>> alice brown 38 3.65
>> alice brown 44 3.82
>> alice brown 59 2.94
>> alice brown 61 2.02
>> alice brown 66 1.57
>> alice brown 69 3.04
>> alice brown 70 1.27
>> alice brown 73 1.75
>> alice brown 73 3.36
>> alice carson 19 1.88
>> alice carson 22 2.82
>> ...
>>
Re: [pig-0.10.0 e2e test]: some tests failed with "Sort check failed" error
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
I'm just curious, why do you expect pig 0.10 tests to succeed on 0.9.2?
D
On Mon, Aug 6, 2012 at 6:57 AM, lulynn_2008 <lu...@163.com> wrote:
> Hi All,
> I am running pig-0.10.0 e2e test with pig-0.9.2 and hadoop-1.0.3. There are 12 tests failed with "Sort check failed" error.
>
> I list Order_6 as a example:
> Error info:
> Going to run sort check command: sort -cs -t +0 -1 ./out/pigtest/pig/pig.1344259341/Order_6.out/out_original
> /bin/sort: ./out/pigtest/pig/pig.1344259341/Order_6.out/out_original:2: disorder: alice allen 36 2.27
> Sort check failed
>
> Related files:
> studenttab10k file:
> ulysses thompson 64 1.90
> katie carson 25 3.65
> luke king 65 0.73
> holly davidson 57 2.43
> fred miller 55 3.77
> holly white 43 0.24
> luke steinbeck 51 1.14
> nick underhill 31 2.46
> holly davidson 59 1.26
> calvin brown 56 0.72
> rachel robinson 62 2.25
> tom carson 35 0.56
> tom johnson 72 0.99
> irene garcia 54 1.06
> oscar nixon 39 3.60
> holly allen 32 2.58
> oscar hernandez 19 0.05
> alice ichabod 65 2.25
> wendy thompson 30 2.39
> priscilla hernandez 73 0.23
> gabriella van buren 68 1.32
> yuri thompson 42 3.65
> yuri laertes 60 1.16
> sarah young 23 2.76
> zach white 32 0.20
> nick van buren 68 1.75
> xavier underhill 41 1.51
> bob ichabod 56 2.81
> zach steinbeck 61 2.22
> alice garcia 42 2.03
> jessica king 29 3.61
> calvin nixon 37 0.30
> fred polk 66 3.69
> bob zipper 40 0.28
> alice young 75 0.31
> nick underhill 37 1.65
> mike white 57 0.69
> calvin ovid 41 3.02
> fred steinbeck 47 3.57
> sarah ovid 65 0.00
> wendy nixon 63 0.6
>
> Order_6.pig file:
> a = load '/user/pig/tests/data/singlefile/studenttab10k';
> c = order a by $0;
> store c into '/user/pig/out/biadmin.1344259341/Order_6.out';
>
> out.original file;
> alice allen 74 0.02
> alice allen 36 2.27
> alice allen 20 2.47
> alice allen 72 1.75
> alice allen 42 2.46
> alice allen 43 0.88
> alice allen 27 1.95
> alice allen 74 2.40
> alice allen 72 3.63
> alice allen 46 3.97
> alice allen 77 2.52
> alice allen 68 3.51
> alice allen 34 1.23
> alice allen 51 1.08
> alice allen 68 3.39
> alice allen 38 1.80
> alice allen 38 0.81
> alice allen 45 2.80
> alice allen 19 1.93
> alice brown 59 2.94
> alice brown 66 1.57
> alice brown 73 1.75
> alice brown 69 3.04
> ...
>
> out_sorted:
> alice allen 19 1.93
> alice allen 20 2.47
> alice allen 27 1.95
> alice allen 34 1.23
> alice allen 36 2.27
> alice allen 38 0.81
> alice allen 38 1.80
> alice allen 42 2.46
> alice allen 43 0.88
> alice allen 45 2.80
> alice allen 46 3.97
> alice allen 51 1.08
> alice allen 68 3.39
> alice allen 68 3.51
> alice allen 72 1.75
> alice allen 72 3.63
> alice allen 74 0.02
> alice allen 74 2.40
> alice allen 77 2.52
> alice brown 18 0.68
> alice brown 21 2.57
> alice brown 27 3.89
> alice brown 38 3.65
> alice brown 44 3.82
> alice brown 59 2.94
> alice brown 61 2.02
> alice brown 66 1.57
> alice brown 69 3.04
> alice brown 70 1.27
> alice brown 73 1.75
> alice brown 73 3.36
> alice carson 19 1.88
> alice carson 22 2.82
> ...
>