You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Hive QA (JIRA)" <ji...@apache.org> on 2014/11/26 02:14:12 UTC
[jira] [Commented] (HIVE-8836) Enable automatic tests with remote
spark client [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225535#comment-14225535 ]
Hive QA commented on HIVE-8836:
-------------------------------
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12683721/HIVE-8836.5-spark.patch
Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/437/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/437/console
Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-437/
Messages:
{noformat}
**** This message was trimmed, see log for full details ****
Reverted 'ql/src/test/results/clientpositive/spark/load_dyn_part10.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/stats5.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin_7.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/ppd_join3.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoin.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/mapjoin_distinct.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join12.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin_16.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/input17.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/ppd_join_filter.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join35.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join_cond_pushdown_4.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucketmapjoin2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/mapjoin1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/insert_into1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/mapjoin_subquery2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoinopt16.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin9.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby11.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/column_access_stats.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join21.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin_25.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join_merging.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join_map_ppr.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/stats0.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin_2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_sortmerge_join_12.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/ppd_multi_insert.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join9.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoin_union_remove_1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/input12.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join30.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join6.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/mapjoin_hook.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/union_remove_19.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/subquery_multiinsert.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoinopt11.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/mapreduce2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/index_auto_self_join.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join_nullsafe.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/union_remove_6.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join_merge_multi_expressions.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/vector_decimal_mapjoin.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join16.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin_20.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucket3.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/ppd_outer_join5.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/vector_mapjoin_reduce.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/semijoin.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/stats_partscan_1_23.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoinopt20.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/merge2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join4.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucket_map_join_tez2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join25.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join_cond_pushdown_unqual4.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/subquery_in.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/sort_merge_join_desc_6.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby_sort_skew_1_23.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucketmapjoin8.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/union_remove_1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join11.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/parallel.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucketmapjoin12.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoinopt5.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/load_dyn_part5.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/sample6.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join27.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/cross_join.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join_filters_overlap.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby_sort_1_23.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/ppd_join4.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join20.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/mapjoin_subquery.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join13.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/sort_merge_join_desc_1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/limit_pushdown.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin_17.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/input18.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join36.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucket_map_join_1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucketmapjoin3.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/insert_into2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/multi_insert_gby.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoinopt17.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/sample1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join22.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/vector_left_outer_join.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby_multi_single_reducer.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/stats1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin_3.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_sortmerge_join_13.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/union30.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoin_union_remove_2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/input13.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join31.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join7.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoinopt12.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join32_lessSize.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/ctas.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/union_remove_7.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join17.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/stats10.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin_21.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucket4.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/multigroupby_singlemr.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join40.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby7_noskew.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/subquery_exists.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join5.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/list_bucket_dml_2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join26.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/union18.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join19.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/sort_merge_join_desc_7.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/union_remove_15.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/mapjoin_test_outer.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/input1_limit.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/multi_insert.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby8.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucketmapjoin9.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/union_remove_2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join12.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucketmapjoin_negative2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucketmapjoin13.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/ppd_outer_join1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoinopt6.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby3_map.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/sample7.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join28.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/union_remove_24.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/temp_table.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join0.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/stats7.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby_complex_types_multi_single_reducer.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/ppd_join5.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join21.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join14.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/sort_merge_join_desc_2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/union_remove_10.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin_18.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join37.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby3.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucket_map_join_2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/bucketmapjoin4.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/insert_into3.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoinopt18.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join30.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoinopt1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/load_dyn_part1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/sample2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join_reorder2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join23.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join_thrift.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/stats2.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin_4.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/disable_merge_for_bucketing.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/union.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby7_map_multi_single_reducer.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin_13.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby_complex_types.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/mergejoins.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/ppd_gby_join.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/input14.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join32.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join_alt_syntax.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join_cond_pushdown_1.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/auto_join8.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/skewjoinopt13.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/groupby3_map_multi_distinct.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join_star.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/parallel_join0.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/union_remove_8.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/smb_mapjoin_22.q.out'
Reverted 'ql/src/test/results/clientpositive/spark/join41.q.out'
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkProcContext.java'
++ svn status --no-ignore
++ egrep -v '^X|^Performing status on external'
++ awk '{print $2}'
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/0.20S/target shims/0.23/target shims/aggregator/target shims/common/target shims/common-secure/target shims/scheduler/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target itests/target itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-unit-hadoop2/target itests/hive-minikdc/target itests/hive-unit/target itests/custom-serde/target itests/util/target itests/qtest-spark/target hcatalog/target hcatalog/core/target hcatalog/streaming/target hcatalog/server-extensions/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target accumulo-handler/target hwi/target common/target common/src/gen spark-client/target contrib/target service/target serde/target beeline/target cli/target odbc/target ql/dependency-reduced-pom.xml ql/target
+ svn update
Fetching external item into 'hcatalog/src/test/e2e/harness'
External at revision 1641742.
At revision 1641742.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12683721 - PreCommit-HIVE-SPARK-Build
> Enable automatic tests with remote spark client [Spark Branch]
> --------------------------------------------------------------
>
> Key: HIVE-8836
> URL: https://issues.apache.org/jira/browse/HIVE-8836
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Affects Versions: spark-branch
> Reporter: Chengxiang Li
> Assignee: Rui Li
> Labels: Spark-M3
> Fix For: spark-branch
>
> Attachments: HIVE-8836.1-spark.patch, HIVE-8836.2-spark.patch, HIVE-8836.3-spark.patch, HIVE-8836.4-spark.patch, HIVE-8836.5-spark.patch
>
>
> In real production environment, remote spark client should be used to submit spark job for Hive mostly, we should enable automatic test with remote spark client to make sure the Hive feature workable with it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)