You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Rohini Palaniswamy (JIRA)" <ji...@apache.org> on 2017/05/24 23:04:07 UTC
[jira] [Updated] (PIG-5059) Pig 0.16 e2e Types_Order tests failed
with Sort check failed
[ https://issues.apache.org/jira/browse/PIG-5059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rohini Palaniswamy updated PIG-5059:
------------------------------------
Fix Version/s: (was: 0.17.0)
0.18.0
> Pig 0.16 e2e Types_Order tests failed with Sort check failed
> ------------------------------------------------------------
>
> Key: PIG-5059
> URL: https://issues.apache.org/jira/browse/PIG-5059
> Project: Pig
> Issue Type: Bug
> Components: e2e harness
> Affects Versions: 0.16.0
> Environment: Ubuntu 14.04
> Mac OS
> CentOS 7.1
> Reporter: Konstantin Harasov
> Fix For: 0.18.0
>
>
> Env: core 5.2, centOS 7.1
> pig: pig-0.16
> Pig 0.16 e2e tests Types_Order_1,2,3,4,11,12,13,14,15,16 failed because of Sort check failed.
> test-base:
> [exec] =========================================================
> [exec] LOGGING RESULTS TO /opt/pig/pig-0.16/test/e2e/pig/testdist/out/log/test_harnesss_1478952742
> [exec] =========================================================
> [exec] Results so far, PASSED: 0 FAILED: 1 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 0 FAILED: 2 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 0 FAILED: 3 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 0 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 1 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 2 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 3 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 4 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 5 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 5 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 6 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 7 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 8 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 9 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Results so far, PASSED: 6 FAILED: 10 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> [exec] Final results , PASSED: 6 FAILED: 10 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
> BUILD FAILED
> TEST: Types_Order_1
> sort ./out/pigtest/-1478952742-nightly.conf/Types_Order_1_benchmark.out/out_original
> test cksum: 1595601925 208685
> benchmark cksum: 1595601925 208685
> Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_1.out/out_original
> /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_1.out/out_original:27: disorder: 18
> Sort check failed
> TEST: Types_Order_2
> Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_2.out/out_original
> /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_2.out/out_original:23: disorder: zach young 3.34
> Sort check failed
> TEST: Types_Order_3
> Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_3.out/out_original
> /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_3.out/out_original:27: disorder: 18
> Sort check failed
> TEST: Types_Order_4
> Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_4.out/out_original
> /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_4.out/out_original:23: disorder: zach young 3.34
> Sort check failed
> TEST: Types_Order_11
> Going to run sort check command: sort -cs -t -k 3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_11.out/out_original
> /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_11.out/out_original:731: disorder: oscar underhill 58 0.1
> Sort check failed
> TEST: Types_Order_12
> Going to run sort check command: sort -cs -t -k 3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_12.out/out_original
> /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_12.out/out_original:4: disorder: irene johnson 61 3.99
> Sort check failed
> TEST: Types_Order_13
> Going to run sort check command: sort -cs -t -k 3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_13.out/out_original
> /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_13.out/out_original:731: disorder: oscar underhill 58 0.1
> Sort check failed
> TEST: Types_Order_14
> Going to run sort check command: sort -cs -t -k 3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_14.out/out_original
> /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_14.out/out_original:4: disorder: irene johnson 61 3.99
> Sort check failed
> TEST: Types_Order_15
> Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_15.out/out_original
> /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_15.out/out_original:27: disorder: 18
> Sort check failed
> TEST: Types_Order_16
> Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_16.out/out_original
> /bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_16.out/out_original:23: disorder: zach young 3.34
> Sort check failed
> I did some research and create one assumption about the cause of the error:
> ROOT-CAUSE:
> In the process of investigation it became clear the following: when we store data in pig, we assign them the data type. This leads to the fact that the file size changed (probably because of the removal of extra delimiters). As a result of subsequent actions sorting is carried out already with the changed data (different file size). In bash (on Cent OS, Ubuntu and Mac OS), in turn, we do not have the ability to work with data types, and when we perform any sorting, we make them with the same "raw data". As a result, the check at the end of the test is incorrect because the files that are compared have different size.
> We also have different results during sorting on Cent OS/Ubuntu and Mac OS. All these inaccuracies lead to failure in sort check.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)