You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Nandor Kollar (JIRA)" <ji...@apache.org> on 2017/04/04 07:59:42 UTC
[jira] [Commented] (PIG-5212) SkewedJoin_6 is failing on Spark
[ https://issues.apache.org/jira/browse/PIG-5212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954752#comment-15954752 ]
Nandor Kollar commented on PIG-5212:
------------------------------------
[~kexianda] could you please help with this issue?
> SkewedJoin_6 is failing on Spark
> --------------------------------
>
> Key: PIG-5212
> URL: https://issues.apache.org/jira/browse/PIG-5212
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: Nandor Kollar
> Fix For: spark-branch
>
>
> result are different:
> {code}
> diff <(head -20 SkewedJoin_6_benchmark.out/out_sorted) <(head -20 SkewedJoin_6.out/out_sorted)
> < alice allen 19 1.930 alice allen 27 1.950
> < alice allen 19 1.930 alice allen 34 1.230
> < alice allen 19 1.930 alice allen 36 2.270
> < alice allen 19 1.930 alice allen 38 0.810
> < alice allen 19 1.930 alice allen 38 1.800
> < alice allen 19 1.930 alice allen 42 2.460
> < alice allen 19 1.930 alice allen 43 0.880
> < alice allen 19 1.930 alice allen 45 2.800
> < alice allen 19 1.930 alice allen 46 3.970
> < alice allen 19 1.930 alice allen 51 1.080
> < alice allen 19 1.930 alice allen 68 3.390
> < alice allen 19 1.930 alice allen 68 3.510
> < alice allen 19 1.930 alice allen 72 1.750
> < alice allen 19 1.930 alice allen 72 3.630
> < alice allen 19 1.930 alice allen 74 0.020
> < alice allen 19 1.930 alice allen 74 2.400
> < alice allen 19 1.930 alice allen 77 2.520
> < alice allen 20 2.470 alice allen 27 1.950
> < alice allen 20 2.470 alice allen 34 1.230
> < alice allen 20 2.470 alice allen 36 2.270
> ---
> > alice allen 27 1.950 alice allen 19 1.930
> > alice allen 27 1.950 alice allen 20 2.470
> > alice allen 27 1.950 alice allen 27 1.950
> > alice allen 27 1.950 alice allen 34 1.230
> > alice allen 27 1.950 alice allen 36 2.270
> > alice allen 27 1.950 alice allen 38 0.810
> > alice allen 27 1.950 alice allen 38 1.800
> > alice allen 27 1.950 alice allen 42 2.460
> > alice allen 27 1.950 alice allen 43 0.880
> > alice allen 27 1.950 alice allen 45 2.800
> > alice allen 27 1.950 alice allen 46 3.970
> > alice allen 27 1.950 alice allen 51 1.080
> > alice allen 27 1.950 alice allen 68 3.390
> > alice allen 27 1.950 alice allen 68 3.510
> > alice allen 27 1.950 alice allen 72 1.750
> > alice allen 27 1.950 alice allen 72 3.630
> > alice allen 27 1.950 alice allen 74 0.020
> > alice allen 27 1.950 alice allen 74 2.400
> > alice allen 27 1.950 alice allen 77 2.520
> > alice allen 34 1.230 alice allen 19 1.930
> {code}
> It looks like the two tables are in wrong order, columns from 'a' should come first, then columns from 'b'. In spark mode this is inverted.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)