You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Nandor Kollar (JIRA)" <ji...@apache.org> on 2017/04/04 07:58:41 UTC
[jira] [Created] (PIG-5212) SkewedJoin_6 is failing on Spark
Nandor Kollar created PIG-5212:
----------------------------------
Summary: SkewedJoin_6 is failing on Spark
Key: PIG-5212
URL: https://issues.apache.org/jira/browse/PIG-5212
Project: Pig
Issue Type: Sub-task
Components: spark
Reporter: Nandor Kollar
Fix For: spark-branch
result are different:
{code}
diff <(head -20 SkewedJoin_6_benchmark.out/out_sorted) <(head -20 SkewedJoin_6.out/out_sorted)
< alice allen 19 1.930 alice allen 27 1.950
< alice allen 19 1.930 alice allen 34 1.230
< alice allen 19 1.930 alice allen 36 2.270
< alice allen 19 1.930 alice allen 38 0.810
< alice allen 19 1.930 alice allen 38 1.800
< alice allen 19 1.930 alice allen 42 2.460
< alice allen 19 1.930 alice allen 43 0.880
< alice allen 19 1.930 alice allen 45 2.800
< alice allen 19 1.930 alice allen 46 3.970
< alice allen 19 1.930 alice allen 51 1.080
< alice allen 19 1.930 alice allen 68 3.390
< alice allen 19 1.930 alice allen 68 3.510
< alice allen 19 1.930 alice allen 72 1.750
< alice allen 19 1.930 alice allen 72 3.630
< alice allen 19 1.930 alice allen 74 0.020
< alice allen 19 1.930 alice allen 74 2.400
< alice allen 19 1.930 alice allen 77 2.520
< alice allen 20 2.470 alice allen 27 1.950
< alice allen 20 2.470 alice allen 34 1.230
< alice allen 20 2.470 alice allen 36 2.270
---
> alice allen 27 1.950 alice allen 19 1.930
> alice allen 27 1.950 alice allen 20 2.470
> alice allen 27 1.950 alice allen 27 1.950
> alice allen 27 1.950 alice allen 34 1.230
> alice allen 27 1.950 alice allen 36 2.270
> alice allen 27 1.950 alice allen 38 0.810
> alice allen 27 1.950 alice allen 38 1.800
> alice allen 27 1.950 alice allen 42 2.460
> alice allen 27 1.950 alice allen 43 0.880
> alice allen 27 1.950 alice allen 45 2.800
> alice allen 27 1.950 alice allen 46 3.970
> alice allen 27 1.950 alice allen 51 1.080
> alice allen 27 1.950 alice allen 68 3.390
> alice allen 27 1.950 alice allen 68 3.510
> alice allen 27 1.950 alice allen 72 1.750
> alice allen 27 1.950 alice allen 72 3.630
> alice allen 27 1.950 alice allen 74 0.020
> alice allen 27 1.950 alice allen 74 2.400
> alice allen 27 1.950 alice allen 77 2.520
> alice allen 34 1.230 alice allen 19 1.930
{code}
It looks like the two tables are in wrong order, columns from 'a' should come first, then columns from 'b'. In spark mode this is inverted.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)