You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Konstantin Harasov (JIRA)" <ji...@apache.org> on 2016/11/15 13:45:59 UTC
[jira] [Created] (PIG-5058) Pig 0.16 e2e Types_Order tests failed
with Sort check failed
Konstantin Harasov created PIG-5058:
---------------------------------------
Summary: Pig 0.16 e2e Types_Order tests failed with Sort check failed
Key: PIG-5058
URL: https://issues.apache.org/jira/browse/PIG-5058
Project: Pig
Issue Type: Bug
Components: e2e harness
Environment: CentOS 7.1
Ubuntu 14.04
Mac OS 10.11.5 (15F34)
Reporter: Konstantin Harasov
Env: core 5.2, centOS 7.1
pig: pig-0.16
Pig 0.16 e2e tests Types_Order_1,2,3,4,11,12,13,14,15,16 failed because of Sort check failed.
test-base:
[exec] =========================================================
[exec] LOGGING RESULTS TO /pig/pig-0.16/test/e2e/pig/testdist/out/log/test_harnesss_1478952742
[exec] =========================================================
[exec] Results so far, PASSED: 0 FAILED: 1 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 0 FAILED: 2 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 0 FAILED: 3 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 0 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 1 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 2 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 3 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 4 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 5 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 6 FAILED: 4 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 6 FAILED: 5 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 6 FAILED: 6 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 6 FAILED: 7 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 6 FAILED: 8 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 6 FAILED: 9 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Results so far, PASSED: 6 FAILED: 10 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
[exec] Final results , PASSED: 6 FAILED: 10 SKIPPED: 0 ABORTED: 0 FAILED DEPENDENCY: 0
BUILD FAILED
TEST: Types_Order_1
sort ./out/pigtest-1478952742-nightly.conf/Types_Order_1_benchmark.out/out_original
test cksum: 1595601925 208685
benchmark cksum: 1595601925 208685
Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n ./out/pigtest/1478952742-nightly.conf/Types_Order_1.out/out_original
/bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_1.out/out_original:27: disorder: 18
Sort check failed
TEST: Types_Order_2
Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_2.out/out_original
/bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_2.out/out_original:23: disorder: zach young 3.34
Sort check failed
TEST: Types_Order_3
Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_3.out/out_original
/bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_3.out/out_original:27: disorder: 18
Sort check failed
TEST: Types_Order_4
Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_4.out/out_original
/bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_4.out/out_original:23: disorder: zach young 3.34
Sort check failed
TEST: Types_Order_11
Going to run sort check command: sort -cs -t -k 3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_11.out/out_original
/bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_11.out/out_original:731: disorder: oscar underhill 58 0.1
Sort check failed
TEST: Types_Order_12
Going to run sort check command: sort -cs -t -k 3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_12.out/out_original
/bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_12.out/out_original:4: disorder: irene johnson 61 3.99
Sort check failed
TEST: Types_Order_13
Going to run sort check command: sort -cs -t -k 3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_13.out/out_original
/bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_13.out/out_original:731: disorder: oscar underhill 58 0.1
Sort check failed
TEST: Types_Order_14
Going to run sort check command: sort -cs -t -k 3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_14.out/out_original
/bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_14.out/out_original:4: disorder: irene johnson 61 3.99
Sort check failed
TEST: Types_Order_15
Going to run sort check command: sort -cs -t -k 1,1 -k 2n,3n ./out/pigtest/-1478952742-nightly.conf/Types_Order_15.out/out_original
/bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_15.out/out_original:27: disorder: 18
Sort check failed
TEST: Types_Order_16
Going to run sort check command: sort -cs -t -k 1r,1r -k 2nr,3nr ./out/pigtest/-1478952742-nightly.conf/Types_Order_16.out/out_original
/bin/sort: ./out/pigtest/-1478952742-nightly.conf/Types_Order_16.out/out_original:23: disorder: zach young 3.34
Sort check failed
REASON:
In the process of investigation it became clear the following: when we store data in pig, we assign them the data type. This leads to the fact that the file size changed (probably because of the removal of extra delimiters). As a result of subsequent actions sorting is carried out already with the changed data (different file size). In bash (on Cent OS, Ubuntu and Mac OS), in turn, we do not have the ability to work with data types, and when we perform any sorting, we make them with the same "raw data". As a result, the check at the end of the test is incorrect because the files that are compared have different size.
We also have different results during sorting on Cent OS/Ubuntu and Mac OS. All these inaccuracies lead to failure in sort check.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)